Motion Kinematics and Dynamics Prediction Using Human Pose Estimation in Videos - Towards Automated, Kinematical Profiling of Swimmers and Ski Jumpers

The international success of world-class athletes depends strongly on the assessment and active improvement of their technique. Camera technology, force sensors, and sophisticated evaluation software have become common tools for compiling sophisticated biomechanical performance profiles, which allow for precisely evaluating the human motion and concluding necessary steps for improving the athletic performance. In this work, we investigate approaches and algorithms for the fully automatic, time-continuous estimation of kinematic and dynamic performance parameters of athletes in sports videos. Therefore, we focus on two specific application scenarios: estimating the motion kinematics of swimmers in a swimming channel and predicting the jump forces of ski jumpers just from video footage. A fundamental concept of the kinematic analysis is the key-pose, a specific, well-defined athlete posture. The precise and continuous identification of key-poses allows for deriving a lot of relevant kinematic parameters: stroke frequency, kick rate, and inner-cyclic interval timings. We show that the identification of a key-pose can be treated as a classification problem and introduce an ensemble of spatiotemporal pose detectors for continuously estimating the occurrence of distinct postures in cyclic motion. While we can not guarantee that these postures represent actual key-poses, they can be used in a simple probabilistic scheme to indirectly infer key-pose occurrences. We furthermore discuss articulate pose estimation for identifying key-poses directly by their defining features. The proposed pose estimator leverages a deep neural network for learning athlete appearance, which enables the articulate estimation of joint locations even in the visually challenging scenario of a swimming channel. With modern pose estimation systems not being immune to estimation errors, we investigate the error susceptibility of two recent systems and derive a taxonomy of the most pertinent estimation errors. Several optimization strategies are discussed, where each method rectifies a different error class, consequently improving joint localization and key-pose identification. At last, we address the question of how the jump forces of a ski jumper can be predicted from video footage. We introduce a neural network build on dilated convolutional layers for predicting a series of force measurements directly from a sequence of athlete poses. An experimental exploration of different architectures indicates the general feasibility of our approach