Look-That-There: Exploiting Gaze in Virtual Reality Interactions

We present a suite of interaction techniques that fundamentally leverages the user’s gaze direction to provide a range of potential benefits over existing techniques such as reduced arm fatigue, more powerful interaction, and more specialized interaction. Because measuring true gaze direction is problematic, we instead approx-imate gaze with a non-linear mapping of head orientation that re-duces neck strain when looking up or down. Given the immaturity of gaze-assisted VR interaction, we chose to prototype interaction designs across a variety of fundamental VR tasks that includes 3D point specification, 3D movement, and environment navigation. For each basic task we created a range of exemplary gaze-based techniques that populate three classifica-tions: “Lazy ” interactions that minimize or obviate hand move-ment, “Helping Hand ” techniques in which gaze augments con-ventional interaction as if with an extra hand, and “Hands Down” manipulations in which gaze offloads the hands so that they can operate specialized devices such as a tablet. Specifically, this paper presents Look-That-There, a technique for moving objects in a virtual environment that does not require hand movement, in addition to gaze-based techniques for selecting menu items, specifying arbitrary 3D points or regions, and orbiting and flying