Interactive Rendering for Projection-Based Augmented Reality Displays

The rapid advances in computing and communications are dramatically changing all aspects of our lives. In particular, sophisticated 3D visualization, display, and interaction technologies are being used to complement our familiar physical world with computer-generated augmentations. These new interaction and display techniques are expected to make our work, learning, and leisure environments vastly more efficient and appealing. Within different application areas, variants of these technologies are currently being pursued in research and development efforts. Virtual Reality (VR) attempts to provide to the user a sense of spatial presence (visual, auditory, and tactile) inside computer-generated synthetic environments. Opaque head-mounted displays (HMDs) have been the traditional VR output devices for many years. A general characteristics of today's HMDs, however, is their imbalanced ratio between heavy optics (that results in cumbersome and uncomfortable devices) and ergonomic devices with a low image quality (i.e., low resolution, small field of view and fixed focal length). To overcome some of their technological and ergonomic shortcomings and to open new application areas, the Virtual Reality community orients itself more and more away from HMDs, towards projection-based spatial displays such as immersive surround screen displays and semi-immersive embedded screen displays. Compared to HMDs, these new devices offer many advantages (e.g., a high and scalable resolution, a large and extendable field of view, an easier eye accommodation, a lower incidence of discomfort due to simulator sickness, light-weight glasses, etc.). In addition, many of them have particular characteristics (such as shape and size) that lend themselves for being employed as metaphors for application-specific functionality, thus making them easier to integrate into our everyday environments. Good examples for this are semi-immersive workbenches whose horizontal display surface lends itself towards supporting a table metaphor for the corresponding Virtual Reality setup. Augmented Reality (AR) superimposes computer-generated graphics onto the user's view of the real world. In contrast to VR, AR allows virtual and real objects to coexist within the same space. Video see-through and optical see-through HMDs are the traditional output technologies, and are still the display devices that are mainly used for Augmented Reality applications. A reorientation of the AR community towards an alternative display technology has not yet happened. Most of the developments and progress made so far are based on very specific applications and technology-tailored employment scenarios. The majority of AR achievements has found few real-world applications. This can partially be attributed to the underlying core technology of AR - including its display devices. As for many other technological domains, AR needs to provide sufficient robustness, functionality and flexibility to find acceptance and to support its seamless integration into our well-established living environments. For instance, many of our real-world items, devices, and tools are developed and tuned for effectively addressing distinct and problem-specific tasks. In contrast to this, many AR applications address specific problems still on an all-purpose technological basis - making use of technologically stagnating devices. A high demand on alternative display technologies exists that improve the shortcomings of traditional devices and open new application areas for AR. Head-attached displays have first been developed in the mid-sixties and still today own the display monopole in AR field. In contrast to VR technology, however, they have barely improved over the previous years and are still far away from being "ultimate displays". The presented projection-based AR (PBAR) concept aims to combine the technological and ergonomic advantages of the well established projection-based Virtual Reality with the application potentials of Augmented Reality. Thus, it strives for opening new application areas for AR. It proposes -taking pattern from the evolution of VR- to detach the display technology from the user to embed it into the real environment instead. However, it is not intended to substitute other display concepts, such as head-attached displays, but rather to present an application-specific alternative. This thesis introduces the projection-based AR concept, presents proof-of-concept prototypes, explaines interactive rendering techniques for PBAR displays, and describes appropriate interaction techniques and applications for PBAR devices