Towards a general theory of 3Dshape perception

Estimating the 3D shape of objects in our environment is one of the most fundamental problems in visual perception, yet it is still rather poorly understood. If you pick up a typical vision text, you'll learn there are many cues to 3D shape, such as shading, texture gradients and specular highlights. A considerable amount of work has studied each cue in isolation and also how various cues can be combined optimally. However, relatively little research has attempted to find commonalities between cues. Here I will argue that a number of seemingly different 3D shape cues could share some common processing strategies. The key insight is that when patterns such as shading or texture are projected from a 3D object into the 2D retinal image, the patterns are systematically distorted in a way that has easily-measurable effects on the local image statistics. The distortions create clearly organized patterns of local image orientation ('orientation fields') that are systematically related to properties of the 3D shape. These orientation fields can be reliably detected by populations of simple filters tuned to different image orientations, as found in V1. I will outline some of the computational benefits of using orientation fields to estimate 3D shape and show through demonstrations and illusions how they can predict both successes and failures of human 3D shape perception. Together these findings suggest that orientation fields could serve as a powerful, 'common currency' for the first stages of 3D shape estimation