Designing familiar augmented and virtual reality environments and interactions through off-the-shelf real-world solutions

Immersive augmented reality systems allow users to superimpose digital content on their physical environments while immersive virtual reality systems simulate virtual spaces that make use of users' knowledge of the physical world. However, in such novel interactive systems, user unfamiliarity can break the illusion of seamless immersion and hinder adoption. We propose to bridge this gap by introducing familiar interactions and environments into immersive augmented and virtual reality. First, to explore familiar interactions, we propose using smartphones, which are familiar and readily available, as companion devices complementing existing interaction modalities for augmented reality head-mounted displays. Current-generation devices primarily support interaction modalities such as gesture, gaze and voice, which are less familiar as interaction modalities and lack precision and tactility, rendering them fatiguing for extended interactions. We leverage user familiarity with smartphone interactions, coupled with their support for precise, tactile touch input, to unlock a broad range of interaction techniques. For instance, by combining the phone's spatial position and orientation and high-resolution touchscreen within an augmented reality environment, we transform the ordinary smartphone into an interior design palette, a rapier or a bow. We describe a prototype implementation and report on the results of a positional accuracy study. Second, we believe that bringing familiar real world environments into virtual spaces can help create immersive and interactive virtual reality experiences. To explore this, we have chosen the real-world library as an expedient pilot and introduce the Library of Apollo, a virtual-reality experience which aims to bring together the connectedness and navigability of digital collections and the familiar browsing experience of physical libraries. An online deployment of our system, together with user surveys and collected session data showed that our users showed a strong preference for our system for book discovery with 41/45 of users saying they are positively inclined to recommend it to a bookish friend.Science, Faculty ofComputer Science, Department ofGraduat