Coordination Strategies for Human Supervisory Control of Robotic Teams

Autonomous mobile sensor teams are crucial to many civilian and military applications. These robotic teams often operate within a larger supervisory system, involving human operators who oversee the mission and analyze sensory data. Here, both the human and the robotic system sub-components, as well as interactions between them, must be carefully considered in designing effective mission coordination strategies. This dissertation explores a series of representative sub-problems relating to the analysis and coordination of both mobile sensors and human operators within supervisory systems. The content herein is presented in three parts: Part I focuses on coordinating operator behavior independently (operator-focused methods), Part II focuses on coordinating mobile-sensor behavior independently (sensor-focused methods), and Part III focuses on jointly coordinating both operator and mobile sensor behavior (joint methods). The content herein is primarily motivated by a particular application in which Unmanned Aerial Vehicles collect visual imagery to be analyzed by a remotely located operator, although many of the results apply to any system of similar architecture. Specifically, with regard to operator-focused methods, Chapter 2 illustrates how physiological sensing, namely eye tracking, may provide aid in modeling operator behavior and assessing the usability of user interfaces. The results of a pilot usability study in which human observers interact with a supervisory control interface are presented, and eye-tracking data is correlated with various usability metrics. Chapter 3 develops robust scheduling algorithms for determining the ordering in which operators should process sensory tasks to both boost performance and decrease variance. A scenario-based, Mixed-Integer Linear Program (MILP) framework is presented, and is assessed in a series of numerical studies. With regard to sensor-focused methods, Chapters 4 and 5 consider two types of supervisory surveillance missions:Chapter 4 develops a cloud-based coverage strategy for persistent surveillance of planar regions. The scheme operates in a dynamic environment, only requiring sporadic, unplanned data exchanges between a central cloud and the sensors in the field. The framework is shown to provide collision avoidance and, in certain cases, produce convergence to a Pareto-optimal coverage configuration. In chapter 5, a heuristic routing scheme is discussed to produce Dubins tours for persistent surveillance of discrete targets, each with associated visibility and dwell-time constraints. Under some assumptions, the problem is posed as a constrained optimization that seeks a minimum-length tour, while simultaneously constraining the time required to reach the first target. A sampling-based scheme is used to approximate solutions to the constrained optimization. This approach is also shown to have desirable resolution completeness properties.Finally, Chapter 6 explores joint methods for coordinating both operator and sensor behavior in the context of a discrete surveillance mission (similar to that of Chapter 5), in which UAVs collect imagery of static targets to be analyzed by the human operator.In particular, a method is proposed to simultaneously construct UAV routes and operator schedules, with the goal of maintaining the operator's task load within a high-performance regime and preventing unnecessary UAV loitering. The full routing/scheduling problem is posed as a mixed-integer (non-linear) program, which can be equivalently represented as a MILP through the addition of auxiliary variables. For scalability, a MILP-based receding-horizon method is proposed to incrementally construct suboptimal solutions to the full optimization problem, which can be extended using a scenario-based approach (similar to that of Chapter 3) to incorporate robustness to operator uncertainty