Seeing and Working with 3D Data in Virtual Reality

University of Minnesota Ph.D. dissertation. 2021. Major: Computer Science. Advisor: Daniel Keefe. 1 computer file (PDF); 126 pages.With recent advances in 3D data capturing technology and simulation, such as motion capture and laser scanning, a big portion of the data analysis process in many disciplines including medical, mechanical engineering, aerospace dynamics, involves examining and working with complex 3D data. Moving on from the traditional 2D keyboard-mouse environment, visualization researchers are seeing great benefits of utilizing the VR technologies to visualize 3D data. However, designing the VR environments to help users exploit these high quality data is challenging for multiple reasons. First, seeing (i.e. visualizing) 3D data requires careful considerations depending on the tasks and types or complexity of the data. Second, although 3D user interfaces theoretically provide great advantages for manipulating 3D data more directly and intuitively, working with 3D data has lack of precision compared to keyboard-mouse interfaces, for example a hand jitter and lack of haptic feedback to support the movements. Motivated by these challenges, this dissertation advances computational tools for seeing, for working, and for combining advanced seeing and working techniques together to enable interactive analyses of complex data. Advances to the science of data visualization address the hard problem of comparing multiple 3D and 4D datasets (i.e., comparative visualization), and include new theory (a taxonomy of fundamental approaches and survey of related work) as well as a real-world application to analyzing multiple phases of ancient architecture in VR. Advances to the science of 3D user interfaces address the long-standing problem of precise and accurate 3D object manipulation in virtual environments, including an application to improving 2D-3D shape matching, as needed in clinical medical imaging. The dissertation culminates by demonstrating that 3D data analyses that require advanced seeing and working simultaneously are, indeed, more challenging than when these tasks are performed in isolation, and that by putting and evaluating advanced visualizations and advanced interaction techniques together, we can help scientists to see and work with 3D data in virtual reality more effectively, differently, and with new perspectives