Designing Augmented Reality and Virtual Reality Interfaces around Hand Expressivity

Augmented Reality (AR) and Virtual Reality (VR) create exciting new opportunities for people to interact with computing resources and information. Less so exciting is the need for holding hand controllers, which limits applications that demand expressive readily available interactions. In response, prior research has investigated freehand AR/VR input by transforming the user body into an interaction medium. While prior work focuses on having users' hands grasp virtual objects, we propose a new interaction technique to have users' hands become virtual objects through imitation, for example, having a thumb-up hand pose to imitate a joystick. We have created a wide array of interaction designs around this idea to demonstrate its applicability in object retrieval and interactivity. Collectively, we call these interaction designs Hand Interfaces. With a series of user studies comparing Hand Interfaces against various baseline techniques, we have collected quantitative and qualitative feedback which indicated that Hand Interfaces is effective, expressive, and fun to use