1 Comparing Learning Techniques for Hidden Markov Models of Human Supervisory Control Behavior

Models of human behaviors have been built using many different frameworks. In this paper, we make use of Hidden Markov Models (HMMs) applied to human supervisory control behaviors. More specifically, we model the behavior of an operator of multiple heterogeneous unmanned vehicle systems. The HMM framework allows the inference of higher operator cognitive states from observable operator interaction with a computer interface. For example, a sequence of operator actions can be used to compute a probability distribution of possible operator states. Such models are capable of detecting deviations from expected operator behavior as learned by the model. The difficulty with parametric inference models such as HMMs is that a large number of parameters must either be specified by hand or learned from example data. We compare the behavioral models obtained with two different supervised learning techniques and an unsupervised HMM training technique. The results suggest that the best models of human supervisory control behavior are obtained through unsupervised learning. Nomenclature = Probability of going from state i to state j = Maximum likelihood estimate of A = State transition matrix = Probability of observing symbol c in state i = Maximum likelihood estimate of = Observable emission matrix H = Hidden Markov Model = Length of observation sequence s M = Number of observable symbols N = Number of hidden states = t-th observation in sequence s = Set of possible hidden states = Property of being at state i at time t V = Set of possible observables = Forward probability = Backward probability = Viterbi backtrack pointer = Probability of the observation sequence and that the state at time t is i = Likelihood of model H = Initial probability of being in state i = Initial probability vector = Probability of the observation sequence and that the state at time t and t+1 are i and j = Smoothed estimator for = Learning rate for all ’s = Learning rate for all ’s = Smoothed estimator for I