Vision-Based Control of Micro–Air–Vehicles: Progress and Problems In Estimation

Abstract — Vision-based control of agile autonomous vehicles in complicated 3-D environments requires fundamental and ground breaking innovations in multiple, related disciplines. These disciplines include control theory, vision processing, signal processing, sensor development, micro-computer tech-nology and the design and instrumentation of micro-air-vehicles (MAVs). Extremely agile small vehicles with acute situational awareness are required for flying through complex environments such as urban canyons confined by buildings, trees,... etc. Vision-based navigation of such vehicles in the neighborhood of ground vehicles, civilians, as well as in poor weather likewise requires a host of innovations in robust vision estimation. Robust vision estimation includes tasks such as feature point extraction, feature point tracking, image registra-tion, segmentation, object detection and object identification. Control of these agile autonomous vehicles requires innovative control methodologies that synthesize image plane inputs in real-time. This problem can be construed as a feedback control problem with millions of raw input channels, the pixels in the image plane. Observation aggregation techniques, rigorously stable “visual servoing ” methods, robust sensor fusion methods and fundamental theoretical studies of the controllability and observability of such flight systems are just some of the control theoretic issues that are currently lacking. This paper outlines progress and several open problems in vision-based estimation for MAVs. It introduces and summarizes several critical technologies that are currently being applied to this overall objective. I