ALLFlight - Imaging sensors fused with Ladar data for moving obstacle detection

Supporting a helicopter pilot during landings and takeoffs in a degraded visual environment (DVE) is one of the challenges within DLR's project ALLFlight (Assisted Low Level Flight and Landing on Unprepared Landing Sites). Different types of sensors (TV, Infrared, mmW radar and Ladar) are mounted onto DLR’s research helicopter FHS (flying helicopter simulator) for gathering different sensor data of the surrounding world. A high performance onboard computer cluster architecture acquires and fuses all the information to get one single comprehensive description of the outside situation. While TV and IR cameras deliver images with frame rates of 25 Hz or 30 Hz, Ladar and mmW radar provide georeferenced sensor data with only 2 Hz or even less. Extraction of moving objects from the Ladar data stream alone seems to be nearly impossible due to the large time difference of 500ms from frame to frame. Solving this problem, we extract in a first step potential moving objects from the IR/TV images. We connect their movements from frame to frame by a trajectory which is defined in image coordinates. In a next step, we look for Ladar points which are projected onto the same image coordinate frame. Whenever some points within each potential moving object trajectory are hit by a nearby Ladar point, we look for their corresponding world coordinates. As far as this world coordinates do not change from frame to frame the virtual movement within the image coordinates results from helicopter movements only. Such candidates can be neglected. All other potential objects which are hit by the Ladar data each 500ms can easily be analyzed within the world coordinates of the Ladar data. Therefore a quite fast 3D interpretation of these movements can be achieved