Interaction, communication and engagement in hybrid virtual and augmented reality environments

This thesis presents research at the intersection of Extended Reality (XR) and Cultural Heritage - Virtual Heritage (VH). Prior research in VH has used Virtual Reality (VR) for immersive visualisation and Augmented Reality (AR) has been used for enhancing exhibitions. These reality technologies were primarily configured for use within museums, which are limited in terms of physical touch and handling capabilities and remote accessibility. Recent advances in XR and its rapid growth and affordability across various consumer devices are providing new opportunities for cultural heritage to be accessed outside the walls of the museums. However, virtual exhibitions and physical visits are different in many ways. Challenges remain for XR in its ability to emulate physicality and materiality, and the social environments within which interpretation, knowledge and ideas are exchanged. Meanwhile, the diversity of devices that are adopted as a consequence of user preferences necessarily means that technologies developed for users should take into account such a need as audiences are central and core to the activities of cultural institutions. In view of such a need, XR presents many challenges that are unsolved, one aspect of which is related to the concept of Extended Reality itself in its broad spectrum between physical reality and virtual reality. As users adopt their preferred devices within the XR spectrum, it has become essential to investigate the intersection of such use if communication within the spectrum is to be made possible. In addressing the need and challenge, this research designed a Hybrid Virtual and Augmented Reality (HVAR) environment and evaluated what could work if users were to socialise between the spectrum of realities. This research presents two frameworks that conceptualise virtual objects and virtual exhibitions for the connections between the spectrum of realities. Three technological developments were made for experimenting with hybridity in terms of interaction, communication and engagement for users with different XR devices. The initial experimental study indicated that users prefer to have control of movement and object information over guided tours and automatically triggered information, providing insights into the interaction design in virtual exhibitions. Follow up studies found that user acceptance of HVAR is strongly affected by social influences, and their communication within HVAR can be explained through the understanding of perceived presence. Building upon these, this research further investigated factors influencing user engagement within HVAR with an innovated objective measure of engagement using a wearable electroencephalogram (EEG) device combined with user activity monitoring at system runtime, suggesting the effects of object interactivity, User-Generated Content (UGC) and avatar proximity on user engagement. This research innovates the use of Hybrid Virtual and Augmented Reality environments. It has implications for the future design, evaluation and application of XR in various domains that necessitates sociality