How image statistics drive shape-from-texture and shape-from-specularity

We present a new visual cue that allows the visual system to solve two key perceptual problems under a large range of circumstances. The first problem is our ability to distinguish specular reflections from texture markings. The second problem is the estimation of 3D object shape from monocular images. Textures and specular reflections both lead to stochastic patterns in images. How can we tell them apart? Recently we have argued that textures and reflections have different statistical properties (e.g. specular reflections of the real world have heavily skewed pixel histograms). However, there is an additional cue, which results from the way that patterns are distorted by 3D shape. As a textured plane is oriented away from frontoparallel, the image of the texture becomes compressed. This provides a cue for 3D shape: if the visual system can measure the compression of the texture at each image location, it can recover local orientation and thus shape. We argue that specular reflections can be treated a bit like textures, because they also lead to stochastic image patterns with well-conserved statistics. When the world is reflected in a specular surface, the reflection is distorted by the shape of the object. The pattern of distortion is a function of the 3D shape, just as it is with textures. Crucially, however, for specularities the compression is a function of surface curvature as well as orientation. Hence, the mapping from image compression to 3D shape follows different rules for specular vs. textured surfaces. We call the pattern of compressions across an image the ‘texture trajectory’. Texture trajectories can allow the visual system to distinguish specular reflections from textures, and to estimate 3D shape for both textured and specular objects. The texture trajectory cue is weakest for spheres and planes, and strongest for objects with very different surface curvatures in orthogonal directions (e.g. cylinders). We exploit this to generate some powerful demos