Fossil group origins: V. the dependence of the luminosity function on the magnitude gap

Context. In nature we observe galaxy aggregations that span a wide range of magnitude gaps between the two first-ranked galaxies of a system (\u394m_12). Thus, there are systems with gaps close to zero (e.g., the Coma cluster), and at the other extreme of the distribution, the largest gaps are found among the so-called fossil systems. The observed distribution of magnitude gaps is thought to be a consequence of the orbital decay of M 17 galaxies in massive halos and the associated growth of the central object. As a result, to first order the amplitude of this gap is a good statistical proxy for the dynamical age of a system of galaxies. Fossil and non-fossil systems could therefore have different galaxy populations that should be reflected in their luminosity functions. Aims: In this work we study, for the first time, the dependence of the luminosity function parameters on \u394m_12 using data obtained by the fossil group origins (FOGO) project. Methods: We constructed a hybrid luminosity function for 102 groups and clusters at z 64 0.25 using both photometric data from the SDSS-DR7 and redshifts from the DR7 and the FOGO surveys. The latter consists of ~1200 new redshifts in 34 fossil system candidates. We stacked all the individual luminosity functions, dividing them into bins of \u394m12, and studied their best-fit Schechter parameters. We additionally computed a "relative" luminosity function, expressed as a function of the central galaxy luminosity, which boosts our capacity to detect differences - especially at the bright end. Results: We find trends as a function of \u394m_12 at both the bright and faint ends of the luminosity function. In particular, at the bright end, the larger the magnitude gap, the fainter the characteristic magnitude M 17. The characteristic luminosity in systems with negligible gaps is more than a factor three brighter than in fossil-like ones. Remarkably, we also find differences at the faint end. In this region, the larger the gap, the flatter the faint-end slope \u3b1. Conclusions: The differences found at the bright end support a dissipationless, dynamical friction-driven merging model for the growth of the central galaxy in group- and cluster-sized halos. The differences in the faint end cannot be explained by this mechanism. Other processes - such as enhanced tidal disruption due to early infall and/or prevalence of eccentric orbits - may play a role. However, a larger sample of systems with \u394m_12> 1.5 is needed to establish the differences at the faint end