Real-time physiological measure and feedback of workload

Understanding and identifying individuals’ capabilities and limitations has always been a challenge within work contexts, but its importance cannot be underestimated.\ud \ud Humans have a limited mental capacity [142], which means that they can only perform a nite set of tasks at any given period of time. Identifying these limitations is a key factor in the reduction and prevention of what is referred to as Mental Workload Overload. These measures are used in research and industry to evaluate the interaction of users with new systems and tasks. Current techniques involve asking users to subjectively assess and self report their levels of workloads using techniques and questionnaires such as NASA-TLX and Instantaneous Self-Assessment (ISA). The subjective measures become highly important when it comes to evaluating more complex systems and tasks, where performance based measures become highly difficult to measure. Even though they are critical for evaluation of these systems, there are certain limitations that cannot be overlooked when using them. Firstly, subjective measures rely on the participants’ ability to judge and report the state throughout the task. This requires not only extra effort from the operator, but also skill and potential training. Secondly, subjective measures, if used in real-time have the potential to interrupt and negatively affect performance; if used post-task, they rely on the operators’ ability to recall what happened during certain moments in the past. Direct physiological measures offer an opportunity to capture workload whilst overcoming these limitations. However, new research is needed to understand how physiological data can be interpreted within the context of theories of mental workload. The research presented in this thesis explores the use of one particular physiological approach, functional Near Infrared Spectroscopy (fNIRS), to assess workload in controlled laboratory settings, to overcome the limitations and complement the use of subjective measures; a measure based on participants’ brain and physiological responses to task demand, that is independent of the task and/or the operator (without interrupting the task or relying on the operator skill to self report).\ud \ud We have examined the reliability of the technique, and significantly extended our understanding of how artefacts affect recordings during both - a Verbal memory task of remembering a seven digit number and a Spacial memory task of remembering a 6x6 shaped grid. Our results showed that artefacts have a significantly different impact during the two types of tasks, further contributing insights into the existing guidelines of using fNIRS to assess workload during typical human computer interaction evaluation settings. We have further evaluated the sensitivity of the tool and understand the potential implications of using fNIRS as a measure in real-time. Our findings validated fNIRS as a sensitive workload measure, having consistent results in line with subjective measures, confirming a correlation between fNIRS and subjective workload questionnaires NASA-TLX and ISA. Having shown the relationship between fNIRS and workload, the last part of this thesis explores the use of fNIRS as a novel approach to providing users with concurrent feedback of their Mental Workload based on the measurements obtained objectively from fNIRS. We compare this feedback to traditional methods of asking users to self-assess and report their own mental workload during an Air Traffic Controller simulation game. In line with previous work, we con rm that self-reporting methods affect both perceived and actual performance. Furthermore, we found that our objective concurrent feedback technique allowed participants to reflect metacognitively on their Mental Workload during tasks, without reducing either actual or perceived performance.\ud \ud fNIRS showed potential to be a useful and reliable additional channel of information about the user during interaction, without further restricting the user during a typical evaluation settings. We found it sensitive to workload, being able to distinguish between various levels of workload, and with great potential for real time, continuous use during tasks. Finally, we explored a new direction of using fNIRS’s assessment of workload in real time, and we investigated how users can use feedback of their current workload state during tasks. This proved to allow users to think metacognitively about their workload during tasks, without negatively affecting their performance or workload