Visually guided flight in Harris' hawks: a computer vision approach

Animals with well-developed visual systems often use vision as an active sense. This means they can manipulate the position and orientation of their viewpoint to interrogate their visual environment. In this thesis, I investigate active vision in a particularly challenging scenario: birds in manoeuvring flight. I focus on Harris' hawks (Parabuteo unicinctus), which strongly rely on vision to coordinate their manoeuvres. I cover three key challenges of studying birds' gaze strategies in flight. First, I explore the problem of accurately tracking an animal's gaze direction. Marker-based motion capture is the most accurate species-agnostic method to track 3D movements in humans and animals, but commercial systems often underperform when applied to animals. I present a marker-labelling method that solves many of the existing problems, based on an arrangement of markers that is optimal for a custom per-frame labelling algorithm. Second, I explore how to capture the visual scene an animal experiences. I solve this computationally, using a 3D model of the flight environment, and a model of the birds' view during flight. I demonstrate how the proposed method allows us to define alternative gaze strategies for hypothesis testing, and explore different methods of modelling the environment, that adapt to varying levels of geometric complexity. Third, I investigate how we can use the generated synthetic data to analyse the hawks' behaviour. For N = 4 birds, I generate semantic maps of the visual scene the animals experience in flight, for a total of n = 97 perching and obstacle avoidance flights. I characterise the birds' gaze strategies around the landing perch and the obstacles, and discuss possible explanations as to why these strategies may be advantageous. Overall, my thesis demonstrates a novel and reliable method of investigating avian vision in flight and opens up new avenues for data-driven models of animal behaviour