A General Approach for ClosedLoop Registration in AR

Figure 1: Two misregistered frames due to user motion in a projector-based AR experiment are shown in (a) and (c). Our approach observes the augmented imagery and corrects any visible misregistration. The corresponding registered frames are shown in (b) and (d). The typical registration process in augmented reality (AR) consists of three independent consecutive stages: static calibration, dynamic tracking, and graphics overlay. The result is that the real-virtual reg-istration is “open loop”—inaccurate calibration or tracking leads to misregistration that is seen by the users but not the system. To cope with this, we propose a general approach to “close the loop” in the displayed appearance by using the visual feedback of regis-tration for pose tracking to achieve accurate registration. Specifi-cally, a model-based method is introduced to simultaneously track and augment real objects in a closed-loop fashion, where the model is comprised of the combination of the real object to be tracked and the virtual object to be rendered. This method is applicable to paradigms including video-based AR, projector-based AR, and diminished reality. Both qualitative and quantitative experiments are presented to demonstrate the feasibility and effectiveness of our approach