Prototyping Methods for Augmented Reality Interaction

The age of wearable technology devices is upon us. These devices are available in many different form factors including head-mounted displays (HMDs), smartwatches and wristbands. They enable access to information at a glance. They are intended to always be ‘‘on’’, to always be acting and to always be sensing the surrounding environment in order to offer a better interface to the real world. A technology suitable for these kinds of user interfaces (UIs) is augmented reality (AR) due to its ability to merge real and virtual objects. It can be difficult and time consuming to prototype and evaluate this new design space due to components that are undeveloped or not sufficiently advanced. To overcome this dilemma and focus on the design and evaluation of new user interfaces instead, it is essential to be able to quickly simulate undeveloped components of a system to enable the collection of valuable feedback from potential users. The aim of the research presented in this thesis was to develop and evaluate two methods that can be used for prototyping AR interaction. The thesis is based on the four attached papers. Paper 1 presents a Wizard of Oz tool called WozARd and the set of tools it offers. The WozARd device allows the test leader to control the visual, tactile and auditive output that is presented to the test participant. WozARd is also suitable for use in an AR environment where images are overlaid on the smartphone’s camera view or on glasses. The main features that were identified as necessary for simulating AR functionality were: presentation of media such as images, video and sound; navigation and location based triggering; automatically taking photos; capability to log test results; notifications; and the integration of the Sony SmartWatch for interaction possibilities. The study described in Paper 2 is an initial investigation of the capability of the WozARd method to simulate a believable illusion of a real working AR city tour. A user study was carried out by collecting and analyzing qualitative and quantitative data from 21 participants who performed the AR city tour using the WozARd with a HMD and smartwatch. The data analysis focused on seven categories that can have a potential impact on how the WozARd method is perceived by participants: precision, relevance, responsiveness, technical stability, visual fidelity, general user experience, and human operator performance. Overall, the results seem to indicate that the participants perceived the simulated AR city tour as a relatively realistic experience despite a certain degree of technical instability and human operator mistakes. Paper 3 presents a proposed method, called IVAR (Immersive Virtual AR), for prototyping wearable AR interaction in a virtual environment (VE). IVAR was developed in an iterative design process that resulted in a testable setup in terms of hardware and software. Additionally, a basic pilot experiment was conducted to explore what it means to collect quantitative and qualitative data with the proposed prototyping method. The main contribution is that IVAR shows potential to become a useful wearable AR prototyping method, but that several challenges remain before meaningful data can be produced in controlled experiments. In particular, tracking technology needs to improve, both with regards to intrusiveness and precision. The goal of Paper 4 was to apply IVAR to evaluate the four interaction concepts from Paper 3: two for device discovery and two for device interaction implemented in a virtual environment. The four interaction concepts were compared in a controlled experiment. Overall, the results indicate that the proposed interaction concepts were found natural and easy to use. Overall, the research presented in this thesis found the two prototyping methods, the WozARd and the IVAR method, to be useful for prototyping AR interaction but several challenges remain before meaningful data can be produced in controlled experiments. WozARd is flexible in terms of being easy to add new UI, and is sufficiently stable for prototyping an ecosystem of wearable technology devices in outdoor environments, but it relies on a well-trained wizard operator. IVAR is suitable for simulations of more complex scenarios, e.g. since registration and tracking easily can be simulated. However, it has the disadvantage of being static, since users need to sit down and their movements are somewhat limited because they are connected to a computer with cables