Development of a non-invasive motion capture system for swimming biomechanics

Sports researchers and coaches currently have no practical tool that can accurately and rapidly measure the 3D kinematics of swimmers. Established motion capture methods in biomechanics are not well suited for underwater use, either because they i) are not accurate enough (like depth-based systems, or the visual hull), ii) would impair the movement of swimmers (like sensor- and marker-based systems), or iii) are too time consuming (like manual digitisation). The ideal for swimming motion capture would be a markerless motion capture system that only requires a few cameras. Such a system would automatically extract silhouettes and 2D joint locations from the videos recorded by the cameras, and fit a generic 3D body model to these constraints. The main challenge in developing such a system for swimming motion capture lies in the development of algorithms for silhouette extraction and 2D pose detection (i.e., localisation of joints in image coordinates), which need to perform well on images of swimmers—a task that currently available algorithms fail. The aim of this PhD was the development of such algorithms. Existing datasets do not contain images of swimmers, making it impossible to train algorithms that would perform well in this domain. Therefore, during the PhD two datasets of images of swimmers were constructed and hand-labelled: one, called Scylla, for silhouette extraction (3,100 images); and one, called Charybdis, for 2D pose detection (8,000 images). Scylla and Charybdis are the first datasets developed specifically for training algorithms to perform well on images of swimmers. Indeed, using these datasets, two algorithms were developed during this PhD: FISHnet, for silhouette extraction; and POSEidon, for 2D pose detection. The novelty of FISHnet (which outperformed state-of-the-art algorithms on Scylla) lies in its ability to predict outputs at the same resolution as the inputs, allowing it to reconstruct fine-grained silhouettes. The novelty of POSEidon lies in its unique structure, which allows it to directly regress the x and y coordinates of joints without needing heatmaps. POSEidon is almost as accurate as humans at locating the spinal joints of swimmers, which are essential constraints onto which to fit 3D models. Using these two algorithms, researchers will, in the future, be able to assemble a markerless motion capture system for swimming, which will contribute to improving our understanding of swimming biomechanics, as well as providing coaches a tool with which to monitor the technique of swimmers