Exploring the use of brain-sensing technologies for natural interactions

Recent technical innovation in the field of Brain-Computer Interfaces (BCIs) has increased the opportunity for including physical, brain-sensing devices as a part of our day-to-day lives. The potential for obtaining a time-correlated, direct, brain-based measure of a participant's mental activity is an alluring and important development for HCI researchers. In this work, we investigate the application of BCI hardware for answering HCI centred research questions, in turn, fusing the two disciplines to form an approach we name - Brain based Human-Computer Interaction (BHCI). We investigate the possibility of using BHCI to provide natural interaction - an ideal form of HCI, where communication between man-and-machine is indistinguishable from everyday forms of interactions such as Speaking and Gesturing. We present the development, execution and output of three user studies investigating the application of BHCI. We evaluate two technologies, fNIRS and EEG, and investigate their suitability for supporting BHCI based interactions. Through our initial studies, we identify that the lightweight and portable attributes of EEG make it preferable for use in developing natural interactions. Building upon this, we develop an EEG based cinematic experience exploring natural forms of interaction through the mind of the viewer. In studying the viewers response to this experience, we were able to develop a taxonomy of control based on how viewers discovered and exerted control over the experience