A Markov random field formulation for dense photometric stereo : theory, practice and applications

In this thesis, we address the problem of normal reconstruction by photomet-ric stereo using a dense set of photometric images captured at fixed viewpoint. Our method is robust to spurious noises caused by highlight, shadows and non-Lambertian reflections which are problematic to traditional approaches. To si-multaneously recover normal orientations and preserve discontinuities, we model the dense photometric stereo problem into two coupled Markov Random Fields (MRFs): a smooth field for normal orientations, and a spatial line process for normal orientation discontinuities. We propose a very fast tensorial belief propa-gation method to approximate the maximum a posteriori (MAP) solution of the Markov network. Our tensor-based message passing scheme not only improves the normal orientation estimation from one of discrete to continuous, but also drastically reduces the storage requirement and running time. We propose a sim-ple image acquisition system that consists of a spherical mirror, a spotlight and a DV camera to collect photometric samples. Using this, we can infer a dense set of unbiased but noisy photometric data roughly distributed uniformly on the light direction sphere. We present very encouraging results on a wide range of difficult objects to show the efficacy of our approach. We apply our accurate normal map we produced in an application in computer graphics. A robust depth map is inferred from the reconstructed normal map which allows for relighting the scene uniformly regardless distant or point light sources. Unlike other model based approaches where parameter estimation is per-formed, a compact, illumination-adjustable and hardware-friendly representation is proposed to encode the versatile reflectance of a complex scene for relighting at very high frame rate. The dense input is first encoded using the principal component analysis (PCA) , which compactly encodes the captured data into a set of eigenimages with the corresponding relighting coefficients. Both are then transformed into a set of texture maps, thus allowing for real-time, per-pixel table-lookup and multiplication by exploiting the state-of-the-art technology in graphics processing unit (GPU). Our experiments show that good visual quality of the relit images and very high frame rate can be achieved simultaneously