Integration of illumination constraints in deformable models

Problems of three dimensional shape estimation from two dimensional images, such as the well-known shape-from-shading (SFS) problem, are fundamental in Computer Vision. Approaches integrating various optical cues in model-based approaches to 3-D estimation have been very successful recently. This dissertation provides a general methodology for the incorporation of illumination constraints within a physics-based deformable model framework, provided that the illumination laws are differentiable functions of the normal to the surface. This encompasses most reflectance models, from the simple Lambertian model to complex highly nonlinear models. We develop a theory for the integration of nonlinear holonomic constraints, such as illumination constraints, using Lagrange multipliers and Baumgarte stabilization. We describe a fast new method for the computation of constraint based forces, and demonstrate experimental improvements in the solution of SFS. The quality of a model\u27s fit to shading data strongly depends on knowledge of lighting conditions. Our framework is valid even when the illuminant direction is unknown. We present a new method for the simultaneous estimation of illuminant direction and SFS, despite the nonlinear relationship between the lighting direction and surface shape. We solve two tightly coupled problems: (a) improving the fit of the model, and (b) gaining a more accurate estimate of the light position. The deformable model formulation allows for easy fusion of our method with other sources of information. We present a multi-view method for the computation of object shape and reflectance characteristics based on the integration of SFS and stereo, for non-constant albedo and non-uniformly Lambertian surfaces. Based on stereo fitting of the input images, we automatically segment the albedo map (assumed piece-wise constant) using a Minimum Description Length (MDL) based metric, to identify areas suitable for SFS (typically smooth textureless areas). Our SFS algorithm iterates between estimating shape and illumination parameters. We demonstrate our method by applying it to face shape reconstruction, with significant experimental improvement over SFS-only or stereo-only based reconstruction, especially in areas of low texture detail, allowing accurate renderings of the model under different illumination conditions