Autonomous Monocular Obstacle Detection for Avoidance in Quadrotor UAVs

Unmanned Aircraft Systems (UAS) research and development and applications have witnessed unprecedented levels of growth in the past two decades. Although military applications have dominated the market, it is anticipated and expected that civilian and public domain applications will be dominant in the future. Consequently, gradual and timely integration of unmanned aviation into the National Airspace System (NAS) is a real challenge, and roadmaps towards achieving full integration are already in place in the US, Canada, Australia, South Africa and European Union (EU). However, before com- plete integration of manned-unmanned aviation, additional challenging problems need to be addressed and solved, which include, among others, flight control systems, sense- detect-and-avoid and see-and-avoid systems, communication spectrum, standards, un- manned aviation safety and reliability, etc. One, distinct, major bottleneck to achieving full integration of manned-unmanned avi- ation is the mid-air collision avoidance problem. Although the manned aviation collision avoidance framework is fully developed and implemented, and technology standards continue to improve, unmanned-manned and unmanned-unmanned mid-air collision avoidance systems have not reached implementation maturity, and wide or universal acceptance (by Federal authorities, industry and other players). The lack of universally acceptable standards adds to the difficulty of solving the problem, although there exist recommendations for system design and performance requirements. Regardless, solving the mid-air collision avoidance problem is a hard prerequisite to integration into the NAS. This research tackles the specific problem of real-time obstacle detection using monoc- ular vision, which is then used for collision avoidance. Focus is on small-scale, Class-I unmanned aerial vehicles (UAVs) with Maximum Takeoff Weight (MTOW) of less than 25 lbs. This class of UAS constitutes the first family of unmanned systems that will be allowed to fly in (restricted) civilian airspace. Emphasis is put on multi-rotor (quadro- tor) UAVs, because they have demonstrated promise and potential for a wide spectrum of civil and public domain applications due to their (mostly) cost-effectiveness, ability to takeoff and land without requiring any infrastructure (runway), and ability to hover, fly in low altitudes, in confined spaces, close to the ground, ceiling and walls, as well as be- cause of their robust flying agility and flexibility (even when a motor fails, for example). The focus of the research is on introducing and developing an on-board the quadrotor cost-effective monocular vision-based system and support technologies for mid-air ob- stacle detection that allows for: i.) robust detection of obstacles in real-time, while flying, and, ii.) ways in which this detection can be used for obstacle avoidance when moving within the environment. A possible extension of this research will allow for the ability to plan paths, trajectories, create, and store and update environment maps (with obstacles) offline and on-line as the quadrotor flies within the environment. There is a plethora of commercially available multi-rotor/quadrotor prototypes with different functionalities and onboard (visual and nonvisual) sensors. Most of such sys- tems are RC-operated/controlled (by a human operator), while some follow visual navi- gation based on an onboard camera. Regardless, their technology is rather restrictive and proprietary to the manufacturer and the application they have been designed for. In ad- dition, there are many University lab-prototypes, the framework of which is mostly 3-D printed, with off-the-shelf-components (motors, sensors, autopilots) and other dedicated technologies and onboard sensors depending on application requirements. As opposed to most existing prototypes, the methodology and support technologies proposed in this thesis are hardware agnostic (independent of specific hardware config- uration). Although for implementation and testing purposes the ARDrone 2.0 quadro- tor is used, the underlying software library may be used on any prototype quadrotor equipped with a monocular camera. Demonstration of the proposed approach consid- ers static and dynamic environments, single and multiple obstacles, and it is evaluated in realistic simulated world environments using ROS, Gazebo and the TUM Simulator package. Interfaces are developed to allow for portability and easy integration, so that the overall ensemble quadrotor (onboard monocular vision system interface) is modular, portable and independent of specific camera and quadrotor type. Obtained results contribute to: i.) a thorough understanding of advantages, disadvan- tages and applicability limitations of using only monocular vision for obstacle detection and avoidance; ii.) real-time applicability, and, iii.) applicability restrictions due to limita- tions of existing support technologies, i.e., ROS, Gazebo and the TUM Simulator package. A major contribution of the developed system is that it may be used as an open-source ed- ucational tool for students, practitioners and end users who are interested in unmanned aviation