Representation of depth in neural population responses

To perceive the visual world as three-dimensional, the brain has to reconstruct spatial structure from retinal images that are only two-dimensional. This ability is called depth perception, and it relies on a variety of depth cues. Disparity and motion are only two examples of such cues. They were, however, chosen for a computational study of the relation between neural activity and properties of perception, because they are particularly suited for such an investigation: visual neurons encoding both cues are well known and the psychophysics of depth from disparity as well as motion have been studied in great detail. Two fundamental problems in depth vision have been addressed. The first arises from inconsistencies between neural and psychophysical data for the processing and perception of disparity: Disparity-sensitive neurons are a possible neural correlate for stereoscopic depth perception. Their perceptual relevance has, however, been questioned, because they vigorously respond to a class of contrast-inverted stimuli that do not evoke depth percepts. A more elaborate representation of depth from disparity in higher visual areas of the cortex was consequently postulated. In part I of the thesis, it is shown by use of simulation of neural populations that the seeming difference between the activity of individual neurons and perception might evolve from properties of a population code of disparity. Data predict a lack of correlation between disparity and perceived depth for contrast-inverted stimuli. Because of the absence of a parameter determining smooth surfaces, large stimuli should evoke confusing but not depth percepts, while small stimuli should generate depth percepts largely independent of disparity. In part II, the latter predictions from the simulated population of disparity-sensitive neurons are confirmed in a series of psychophysical experiments. Small contrast-inverted stimuli (diameter of less than 3 deg of visual angle) cannot be distinguished from normal ones. They do evoke depth percepts which, however, are not determined by disparity. When contrast-inverted stimuli become larger, they are perceived as being different. Both psychophysical experiments (part II) and simulations (part I) demonstrate, that the absence of depth percepts in contrast-inverted stimuli is already discernible in the early neural representation. Psychophysical and neural data, previously reported, are, accordingly, not in conflict but share one of the crucial properties. Part I and II are not suited to rule out the postulated existence of more elaborate disparity detectors in higher brain centers. While these remain to be verified and are, so far, only hypothetical, this thesis demonstrates that psychophysical depth is represented in the earliest cortical representation of disparity. The second fundamental problem treated relates to the richness of depth cues used by the brain. In natural scenes available depth cues are combined to a uniform consistent depth percept. The neural mechanisms of depth cue integration are largely unknown, although their perceptual properties have been investigated in a great many of psychophysical studies. The integration of disparity and motion forms, to some extent, an exception: neurons sensitive to disparity equally encode motion (and motion direction). They are a potential neural correlate of unified depth percepts from both cues. In part III and IV, the properties of disparity and motion combination in population responses is investigated by use of simulations. Represented depth is largely explained by a linear interaction of the two cues. Non-linear phenomena like sub- threshold summation are, nevertheless, also found and a regression analysis confirms non-linear interactions. The depth representation is robust against various manipulations of either of the depth cues. The absence of one of the cues, in particular, does not seriously hamper depth estimation. The representations of depth from retinal motion that is generated by either rotation or translation are compared. Depth from motion was severely degraded when motion was translatory. This yields, for the first time, an explanation for an irritating transition in the psychophysical combination of disparity and motion found for human observers: When disparity is constantly kept zero, motion influences depth percepts strongly. By contrast, constant non-zero disparity values overrule depth from motion. Eye motion was not measured in these psychophysical experiments, but it is known from other studies that compensatory eye movements in zero disparity displays generate rotatory motion, while the absence of compensation in non-zero disparity displays results in translatory motion geometry. Part III and IV localize a broad range of cue interactions in a known neural correlate in early visual areas. Geometrical properties of cue generation were shown to mediate between contradictory types of cue combination. The model presented establishes, for the first time, a physiologically plausible explanation of psychophysical findings. To summarize, simulations of neural population responses and psychophysical experiments are used to link data from the single neuron level to the level of perception. Crucial properties of perceptual phenomena are found to be represented in populations of visual neurons. This demonstrates the perceptual significance of earliest areas of the visual cortex. While the study could only tackle exemplary questions (even under the limited scope of depth vision) the methodology proved to be a powerful tool. Its application to unsolved problems in early vision appears promising