Lightweight binocular facial performance capture under uncontrolled lighting

Figure 1: Three results of our facial performance capture method for two indoor sequences with fast and expressive motion. Recent progress in passive facial performance capture has shown impressively detailed results on highly articulated motion. How-ever, most methods rely on complex multi-camera set-ups, con-trolled lighting or fiducial markers. This prevents them from being used in general environments, outdoor scenes, during live action on a film set, or by freelance animators and everyday users who want to capture their digital selves. In this paper, we therefore propose a lightweight passive facial performance capture approach that is able to reconstruct high-quality dynamic facial geometry from only a single pair of stereo cameras. Our method succeeds under uncon-trolled and time-varying lighting, and also in outdoor scenes. Our approach builds upon and extends recent image-based scene flow computation, lighting estimation and shading-based refinement al