Usability evaluation of non-immersive, desktop, photo-realistic virtual environments

This study is about the evaluation of non-immersive desktop web-based photo-realistic virtual environments using think-aloud protocol and heuristic evaluation to determine three aims: (1) whether applying the same usability evaluation methodologies result in additional usability guideline categoriesidentified from the Koykka, Ollikainen, Ranta-aho, Milszus, Wasserroth and Friedrich 1999 study; (2) whether think-aloud protocol or usability heuristic evaluation is a better evaluation method for identifying usability problems in desktop, photo-realistic virtual environments; and (3) whether large-scale desktop, photo-realistic virtual environments will have more usability problems than small-scale non-immersive, desktop, photo-realistic virtual environments. The results show that using the Emergent Theme Analysis (ETA), four broad themes were derived: functionality, interaction, appearance and user comments with functionality and interaction broad themes being similar to the categories suggested by Koykka et al. (1999). Furthermore, the results indicate that small-scale non-immersive, desktop, photo-realistic virtual environments had more usability problems than large-scale non-immersive, desktop, photo-realistic virtual environments, contrary to the hypothesis that large-scale VEs will have more usability problems due to its complexity and that the think-aloud protocol derived more themes compared to the heuristic evaluation – suggesting that TAP is a better usability evaluation method than HE in this type of study. However, a combination of the two qualitative methods has identified a greater number of usability problems, supporting the need for triangulation of research methods. The investigation resulted in new design guidelines that will allow for more usable design of non-immersive desktop, photo-realistic virtual environments. Furthermore, the study provides some new areas for future developments of usability evaluation methods for non-immersive desktop, photo-realistic virtual environments.PublishedNon Peer ReviewedAxup, J. 2002, Comparison of Usability Evaluation Methods (UEM), Published on-line at Date Accessed UserDesign.com. Bowman, D. A. 2002, 'Principles for the design of performance-oriented interaction techniques', . Bowman, D. A. & L. F. Hodges (1997) An Evaluation of Techniques for Grabbing and Manipulating Remote Objects in Immersive Virtual Environments. 1997 Symposium on Interactive 3D Graphics, April 1997 pp. 35 - ff. Bowman, D. A., D. B. Johnson & L. F. Hodges (1999) Evaluation of Virtual Environment Interaction Techniques. VRST'99: ACM Symposium on Virtual Reality Software and Technology, University College, London, December 20-22, 1999 pp. 26-33. Chen, J.-L. (2000) A model of wayfinding in virtual environments: Strategies for navigational aids. Dissertation Abstracts International: Section B: the Sciences & Engineering, 60:11-B, pp. 5701, US: Univ Microfilms International. Dalgarno, B. & J. Scott (1999) Usability Problems in Desktop Virtual Environments. OZCHI'99 Interfaces for the Global Community, Charles Sturt, Wagga Wagga, Australia, pp. 134 - 136. Darken, R. P. & J. L. Sibert (1996) Wayfinding Strategies and Behaviours in Large Virtual Environments. Computer-Human Interaction (CHI), . Goldbaum, H. 2003, The Virtual Wroxton Abbey, Published on-line at Date Accessed Oxfordshire, England, Wroxton College. Johnson, C. 1998, On the Problems of Validating DesktopVR Innovative User Interfaces: Multimedia and Multimodal User Interfaces, Wearable Computers and Virtual Reality, Published on-line at Date Accessed Proceedings of the HCI'98 Conference on People and Computers XIII. Koykka, M., R. Ollikainen, M. Ranta-aho, W. Milszus, S. Wasserroth & M. Friedrich (1999) Usability Heuristic Guidelines for 3D Multi-User Worlds. Interfaces for the Global Community, Charles Sturt University, Wagga Wagga, Australia, pp. 52 - 57. Molich, R. & J. Nielsen (1990) Improving a Human-Computer Dialogue. Communications of ACM, 33:3, pp. 338- 348. Nielsen, J. (1994) Usability Inspection Methods. CHI'94, Boston, Massachusetts, USA, April 24-28, 1994 pp. 413-414. Nielsen, J. 1997, 'Usability Testing', in Handbook of Human Factors and Ergonomics, Ed G. Salvendy, John Wiley and Sons, pp. 1543-1568. RichardStrauss 2003, The Villa in Garmisch, Published on-line at Date Accessed Garmisch, Bavaria, Richar Strauus. Sandra, L. 2002, Task Analysis, pp 1-6. Saretto, C. J. (1997) A Survey of Interaction Issues Facing Navigation in Virtual Worlds. CHI 97: Conference on Human Factors in Computing Systems, Los Angeles, USA, April 18-23, 1997 pp. 1-7. Stanney, K. M., M. Mollaghasemi, L. Reeves, R. Breaux & D. A. Graeber (2003) Usability engineering of virtual environments (VEs): identifying multiple criteria that drive effective VE system design. International Journal of Human-Computer Studies, 58:4, pp. 447-481. Sutcliffe, A. & K. Deol Kaur 2000, Evaluating the Usability of Virtual Reality User Interfaces, Published online at Date Accessed Behaviour and Information Technology. Vinson, N. G. (1999) Design Guidelines for Landmark to Support Navigation in Virtual Environments. CHI'99: The CHI is the Limit ACM SIGCHI Conference on Human Factors in Computing Systems, Pittsburgh, PA USA, May 15 - 20, 1999 pp. 278-285. Zhang, Z. W. 2003, Usability Evaluation Methods, Published on-line at Date Accessed Drexel University