Lightweight and Motion Guided Neural Network Algorithms for Onboard Drone Detection and Flight

The last several years have seen a substantial increase in both the recreational and commercial usage of unmanned aerial vehicles, also commonly known as drones. However, alongside the harmless use of drones in recreational flying and commercial applications, easy access to such technology has led to significant instances of drone misuse. From occurrences of shutting down whole airports with an unauthorised drone to instances of flying contraband into prisons, there is strong interest in the measures to prevent malicious drone flight. In this dissertation we investigate drone detection and surveillance measures in the context of a fully autonomous onboard drone detection and pursuit system. While radar is the conventional method to detect the presence of other aircraft, commercially available drones have a much smaller profile and cannot be reliably detected by radar. Visual detection systems are instead the most viable approach to detect drones and while it is possible to use existing convolutional neural networks to detect nearby drones, it does not offer a solution to prevent drones from accessing an area. We therefore propose the solution of a fully autonomous drone which can detect other drones and engage them in a targeted pursuit if they are nearby. While existing convolutional neural network approaches can readily detect drones, these models are often too large to achieve real-time speeds when mounted on small hardware that would fit onboard a drone. To address this challenge, we are motivated by biological systems, specifically the visual processing of insects tuned to hunt small moving prey with near perfect catch rates. With their low resolution blurry vision and a small, computationally constrained brain, these insects exploit the motion of small moving objects to detect their prey. Motivated by this, in this dissertation we therefore focus on Lightweight and Motion Guided Neural Network Algorithms for Onboard Drone Detection and Flight. To leverage the temporal information available in video sequences, we investigate how neural networks process motion by training them against numerous synthetic psychophysical stimuli. We explore scenarios where appearance information is not available and assess the feasibility of using motion information for small object detection. We build multiple novel lightweight temporally-aware networks to detect real-world small moving drones and other small moving objects. Alongside using conventional recurrent units and frame-stacking to capture temporal information, we introduce a novel biologically inspired band-pass filter for lightweight temporal information propagation. To assist with navigation of a drone in pursuit of an unauthorised drone, a lightweight image segmentation algorithm is created. This allows for a drone system in flight to gain a detailed understanding of the types of objects in its environment and aids with deciding the optimal path to pursue an adversarial drone. Motivated by the challenge of implementing a fully autonomous drone detection and pursuit system, we custom build a drone that can fly with onboard neural network accelerating hardware. We deploy one of our custom lightweight neural networks onboard our drone and examine its ability to track and follow an adversarial drone flying in close proximity. In this dissertation we recognise the challenges that the commercial availability of drones poses and the dangers of possible drone misuse. Our contributions of lightweight drone detection and navigation provides a measure to enhance the security of areas that cannot afford the trespassing of unauthorised drones. As the miniaturisation of neural network acceleration hardware improves, we foresee our lightweight neural network architectures being able to be deployed on even smaller devices and drones.Thesis (Ph.D.) -- University of Adelaide, School of Computer and Mathematical Sciences, 202