Fossil group origins: V. the dependence of the luminosity function on the magnitude gap

Context. In nature we observe galaxy aggregations that span a wide range of magnitude gaps between the two first-ranked galaxies of a system (\u394m12). Thus, there are systems with gaps close to zero (e.g., the Coma cluster), and at the other extreme of the distribution, the largest gaps are found among the so-called fossil systems. The observed distribution of magnitude gaps is thought to be a consequence of the orbital decay of M 17 galaxies in massive halos and the associated growth of the central object. As a result, to first order the amplitude of this gap is a good statistical proxy for the dynamical age of a system of galaxies. Fossil and non-fossil systems could therefore have different galaxy populations that should be reflected in their luminosity functions. Aims: In this work we study, for the first time, the dependence of the luminosity function parameters on \u394m12 using data obtained by the fossil group origins (FOGO) project. Methods: We constructed a hybrid luminosity function for 102 groups and clusters at z 64 0.25 using both photometric data from the SDSS-DR7 and redshifts from the DR7 and the FOGO surveys. The latter consists of ~1200 new redshifts in 34 fossil system candidates. We stacked all the individual luminosity functions, dividing them into bins of \u394m12, and studied their best-fit Schechter parameters. We additionally computed a "relative" luminosity function, expressed as a function of the central galaxy luminosity, which boosts our capacity to detect differences - especially at the bright end. Results: We find trends as a function of \u394m12 at both the bright and faint ends of the luminosity function. In particular, at the bright end, the larger the magnitude gap, the fainter the characteristic magnitude M 17. The characteristic luminosity in systems with negligible gaps is more than a factor three brighter than in fossil-like ones. Remarkably, we also find differences at the faint end. In this region, the larger the gap, the flatter the faint-end slope \u3b1. Conclusions: The differences found at the bright end support a dissipationless, dynamical friction-driven merging model for the growth of the central galaxy in group- and cluster-sized halos. The differences in the faint end cannot be explained by this mechanism. Other processes - such as enhanced tidal disruption due to early infall and/or prevalence of eccentric orbits - may play a role. However, a larger sample of systems with \u394m12> 1.5 is needed to establish the differences at the faint end