A Cognitively Informed and Network Based Investigation of Human Neural Activities, Behaviors, and Performance in Human-Autonomy Teaming Tasks

Human-autonomy teams are expected to provide solutions in a wide range of applications, such as human directed search and rescue, hazard containment and mobilization, and space exploration. These teams consist of autonomous agents that coordinate their actions with the human partner to achieve common goals. Despite the advancements of current autonomous systems, it is the human's ability to engage their knowledge and expertise that makes human-autonomy teams especially effective in tasks dominated by dynamic and uncertain conditions. The human and their autonomous teammate should have shared plans and a similar focus of attention. However, studies have shown that a human's miscomprehension of an autonomous system's state, decisions, or course of action can result in misuse or disuse of the agent, causing a reduction in team performance. The aim of this dissertation is to improve human-autonomy team task proficiency by investigating methods to measure changes in human cognitive state as reflected in neurophysiological measures using methods derived from network science. This work is comprised of two primary studies. In the first study, we examined human behaviors and brain activity acquired via electroencephalography (EEG) to probe the interactions between cognitive processes, behaviors, and performance in a human-multiagent team task. We showed that measurable changes in brain activity indicate a higher burden on the cognitive resources associated with visual-spatial reasoning required to estimate a more complex kinematic state of robotic agents. These conclusions were reinforced by complementary behavioral shifts in gaze and pilot inputs. Next, we showed that EEG inter-channel connectivity network metrics distinguish gaze behaviors associated with the attention process more effectively than traditional single-channel features. In the second study we explored the relationship between neurophysiological features and human trust in an autonomous system while performing a team task. Trust prediction models were constructed using a variety of feature types determined from an EEG timeseries. A comparison of model performance between traditional EEG signal powers with inter-channel connectivity network metrics revealed that measures of dynamic changes in synchronous behavior between distant brain regions can capture cognitive activities that predict a human's trust in an autonomous system. We showed that both single-channel powers and network-metrics defined from brain regions associated with reasoning and attention have the greatest impact on trust prediction. In a third study, we explore the interaction between behaviors and performance for subjects of various skills in a manual grinding task. We show that there were observable and distinguishable sensorimotor behaviors associated with two distinct techniques utilized by the individual subjects, and that task performance is affected by these techniques