Shape and Reflectance from Natural Illumination

Abstract. We introduce a method to jointly estimate the BRDF and geometry of an object from a single image under known, but uncontrolled, natural illumina-tion. We show that this previously unexplored problem becomes tractable when one exploits the orientation clues embedded in the lighting environment. Intu-itively, unique regions in the lighting environment act analogously to the point light sources of traditional photometric stereo; they strongly constrain the ori-entation of the surface patches that reflect them. The reflectance, which acts as a bandpass filter on the lighting environment, determines the necessary scale of such regions. Accurate reflectance estimation, however, relies on accurate sur-face orientation information. Thus, these two factors must be estimated jointly. To do so, we derive a probabilistic formulation and introduce priors to address situations where the reflectance and lighting environment do not sufficiently con-strain the geometry of the object. Through extensive experimentation we show what this space looks like, and offer insights into what problems become solv-able in various categories of real-world natural illumination environments.