Orientation fields in the perception of 3D shape and material properties

I present a theoretical analysis that explains how the visual system could solve two key perceptual problems. The rst problem is our ability to distinguish reections from texture markings. The second problem is the estimation of 3D object shape from monocular images. Textures and reections both lead to stochastic patterns in images. How can we tell them apart? We have argued previously [1] that textures and reections have different statistical properties (e.g. specular reections of the real world have heavily skewed pixel histograms). However, there is an additional cue, which results from the way that patterns are distorted by 3D shape. As a textured plane is oriented away from frontoparallel, the image of the texture becomes compressed. This provides a cue for 3D shape: if the visual system can measure the compression of the texture at each image location, it can recover the rst derivative of the surface (i.e. local orientation) and thus shape. I argue that specular reections can be treated a bit like textures, because they also lead to stochastic image patterns with well-conserved statistics. When the world is reected in a specular surface, the reection is distorted by the shape of the object. The pattern of distortion is a function of the 3D shape, just as it is with textures. Crucially, however, for specularities the compression is a function of the second rather than the rst derivative of the surface (i.e. surface curvature as well as orientation). Hence, the mapping from image compression to 3D shape follows different rules for specular vs. textured surfaces. I show that the compressions produced by 3D curvatures reliably lead to characteristic `elds' of orientation energy across the image of a specular surface. These orientation elds are diagnostic of 3D shape but remain surprisingly stable across changes in the scene reected in the surface. Furthermore, I show that these characteristic orientation elds can be easily extracted from the image by populations of linear lters that resemble the oriented receptive elds of V1 cells. I show how orientation elds could allow the visual system to distinguish between reections and textures, even when they are present simultaneously. Finally, I discuss the generalization of these principles to surfaces with arbitrary reectance properties