Depth from Shading, Defocus, and Correspondence Using Light-Field Angular Coherence

Light-field cameras are now used in consumer and in-dustrial applications. Recent papers and products have demonstrated practical depth recovery algorithms from a passive single-shot capture. However, current light-field capture devices have narrow baselines and constrained spatial resolution; therefore, the accuracy of depth recov-ery is limited, requiring heavy regularization and produc-ing planar depths that do not resemble the actual geome-try. Using shading information is essential to improve the shape estimation. We develop an improved technique for local shape estimation from defocus and correspondence cues, and show how shading can be used to further refine the depth. Light-field cameras are able to capture both spatial and angular data, suitable for refocusing. By locally refocusing each spatial pixel to its respective estimated depth, we pro-duce an all-in-focus image where all viewpoints converge onto a point in the scene. Therefore, the angular pixels have angular coherence, which exhibits three properties: photo consistency, depth consistency, and shading consistency. We propose a new framework that uses angular coherence to optimize depth and shading. The optimization framework estimates both general lighting in natural scenes and shad-ing to improve depth regularization. Our method outper-forms current state-of-the-art light-field depth estimation algorithms in multiple scenarios, including real images. 1