Increasing Situation Awareness for a Helicopter Pilot by Data Acquisition and Fusion of Multiple Sensors

After a successful final demonstration of the pilot assistant system developed in the scope of the DLR project PAVE (Pilot Assistant in the Vicinity of Helipads) at the end of 2007, a new project called ALLFlight (Assisted Low Level Flight and Landing on Unprepared Landing Sites) has been started at the beginning of 2008. The main objective is to achieve a safe and effective 24h all weather operation capability of DLR’s research helicopter EC-135 under difficult visual conditions, e.g. in brown-out or white-out situations. This will be realized by providing the pilot assistance with the help of advanced visual and tactile cueing and intelligent control augmentation. Reducing pilot’s workload and increasing his situational and mission awareness are some more objectives within this project. ALLFlight deals with the integration of a full scale enhanced vision sensor suite onto DLR’s research helicopter. This sensor suite consists of a wide set of imaging sensors, such as a colour TV camera, an un-cooled thermal infrared camera, an optical radar scanner and a mmW radar system. The latter two radar systems are equipped with specialized software for experimental modes, such as terrain mapping and ground scanning. In this ALLFlight project DLR’s Institute of Flight Guidance is going to transform the gathered expertise of mostly fixed wing applications in the field of enhanced and synthetic vision research projects for a helicopter application. For fixed wing application enhanced vision (EV) systems - at least the infrared imaging systems - did already enter the phase of industrial line production. Together with the introduction of certified operational rules (e.g. the FAA enhanced flight vision system rule), EV for fixed wing aircraft landing under adverse weather on un-equipped airfields can be regarded technically as present state-of-the-art. Compared to this situation of the fixed wing aircraft, enhanced vision (and synthetic vision) systems for helicopter application are quite rare. Wherever EV concepts for helicopters have been proposed and experimentally realized, it has been mostly a simple adaptation of one single existing imaging or ranging sensor and its combination with any direct visualization and display concept. A compact high performance so-called sensor co-computer system (SCC) has been designed and realized to be able to process and display the huge incoming flood of data from these sensors. This computer system can easily be installed into the helicopter’s experimental electronic cargo bay. A sophisticated, high performance, distributed data acquisition, recording, processing, and fusion software architecture has been developed and implemented during the first project year. This paper explains the architectural software concept and the implementation of the SCC. Some first data replays of gathered sensor data and some concepts of data fusion and display formats will be discussed. During the forthcoming years until 2012 the output will hopefully result into a broader mission potential of the helicopter compared to the present situation whenever a mission cannot be performed or has to be cancelled due to a degraded visual environment