Deep Reflectance Maps

Undoing the image formation process and therefore de-composing appearance into its intrinsic properties is a chal-lenging task due to the under-constraint nature of this in-verse problem. While significant progress has been made on inferring shape, materials and illumination from images only, progress in an unconstrained setting is still limited. We propose a convolutional neural architecture to estimate reflectance maps of specular materials in natural lighting conditions. We achieve this in an end-to-end learning formu-lation that directly predicts a reflectance map from the image itself. We show how to improve estimates by facilitating ad-ditional supervision in an indirect scheme that first predicts surface orientation and afterwards predicts the reflectance map by a learning-based sparse data interpolation. In order to analyze performance on this difficult task, we propose a new challenge of Specular MAterials on SHapes with complex IllumiNation (SMASHINg) using both synthetic and real images. Furthermore, we show the application of our method to a range of image-based editing tasks on real images. 1