Spectroscopy of luminous z > 7 galaxy candidates and sources of contamination in z > 7 galaxy searches

We present three bright z(+)-dropout candidates selected from deep near-infrared (NIR) imaging of the COSMOS 2 deg(2) field. All three objects match the 0.8-8 mu m colors of other published z > 7 candidates but are 3 mag brighter, facilitating further study. Deep spectroscopy of two of the candidates covering 0.64-1.02 mu m with Keck-DEIMOS and all three covering 0.94-1.10 mu m and 1.52-1.80 mu m with Keck-NIRSPEC detects weak spectral features tentatively identified as Ly alpha at z = 6.95 and z = 7.69 in two of the objects. The third object is placed at z similar to 1.6 based on a 24 mu m and weak optical detection. A comparison with the spectral energy distributions of known z < 7 galaxies, including objects with strong spectral lines, large extinction, and large systematic uncertainties in the photometry, yields no objects with similar colors. However, the lambda > 1 mu m properties of all three objects can be matched to optically detected sources with photometric redshifts at z similar to 1.8, so the non-detection in the i(+) and z(+) bands is the primary factor which favors a z > 7 solution. If any of these objects are at z similar to 7, the bright end of the luminosity function is significantly higher at z > 7 than suggested by previous studies, but consistent within the statistical uncertainty and the dark matter halo distribution. If these objects are at low redshift, the Lyman break selection must be contaminated by a previously unknown population of low-redshift objects with very strong breaks in their broadband spectral energy distributions and blue NIR colors. The implications of this result on luminosity function evolution at high redshift are discussed. We show that the primary limitation of z > 7 galaxy searches with broad filters is the depth of the available optical data.Peer reviewed: YesNRC publication: Ye