Coherent intrinsic images from photo collections

Figure 1: Our method leverages the heterogeneity of photo collections to automatically decompose photographs of a scene into reflectance and illumination layers. The extracted reflectance layers are coherent across all views, while the illumination captures the shading and shadow variations proper to each picture. Here we show the decomposition of three photos in the collection. An intrinsic image is a decomposition of a photo into an illumi-nation layer and a reflectance layer, which enables powerful edit-ing such as the alteration of an object’s material independently of its illumination. However, decomposing a single photo is highly under-constrained and existing methods require user assistance or handle only simple scenes. In this paper, we compute intrinsic de-compositions using several images of the same scene under differ-ent viewpoints and lighting conditions. We use multi-view stereo to automatically reconstruct 3D points and normals from which we derive relationships between reflectance values at different lo-cations, across multiple views and consequently different lighting conditions. We use robust estimation to reliably identify reflectance ratios between pairs of points. From these, we infer constraints for our optimization and enforce a coherent solution across multi-ple views and illuminations. Our results demonstrate that this con-strained optimization yields high-quality and coherent intrinsic de-compositions of complex scenes. We illustrate how these decompo-sitions can be used for image-based illumination transfer and tran-sitions between views with consistent lighting