Intelligent Autonomy and Vision Based Obstacle Avoidance for

The paper describes the development and implementation of the Visual Threat Awareness (VISTA) system, its integration with the Multi-layer Architecture for Trajectory Replanning and Intelligent plan eXecution (MATRIX) for autonomous intelligent control of Unmanned Aerial Vehicles (UAV), and performance evaluation of the integrated system through flight tests. The VISTA system generates information on the threats and obstacles in real-time, and passes it on to the MATRIX system that makes mission-related decisions and generates new waypoints and a trajectory that safely avoids the obstacle. The VISTA system combines binocular visual stereo, perceptual organization, graph partitioning and feature tracking for a passive system to enable real-time obstacle detection. Computational stereo performance has progressed such that there now exist several commercial or open source implementations that operate at frame rate, but suffer from well known correspondence errors. We show that introducing a global segmentation step after commodity stereo can increase robustness and leverage existing stereo software. The global segmentation step is based on a graph structure appropriate for collision detection, human vision inspired perceptual organization and graph partitioning using the minimum s-t graph cut. This system has been prototyped using Sarnoff Corp's Acadia I vision processor to enable 640x480@10Hz operation on embedded avionics. We describe VISTA system theory and show proof of concept and flight experiment results of the integrated MATRIX/VISTA system on Georgia Tech's GT-Max autonomous helicopter