A General Approach for ClosedLoop Registration in AR

Tracking and augmentation are usually handled in independent con-secutive stages in augmented reality (AR). The result is that the real-virtual registration is “open loop”—inaccurate tracking leads to misregistration that is seen by the users but not the system. We propose a general approach to “close the loop ” in the displayed appearance by using the visual feedback of registration for track-ing. Specifically, a model-based method is introduced to simulta-neously track and augment real objects in a closed-loop fashion, where the model is comprised of the combination of the real object to be tracked and the virtual object to be rendered. This method is applicable to paradigms including video-based AR, projector-based AR, and diminished reality