Machine Learning gesture detection from pose recognition in video analysis

Recent developments in deep learning and pose recognition make more accessible tracking objects in video sequences. We believe that a useful application of pose recognition system could be in sport analysis field, in particular in the detection of athletes' gestures. In this work we use a human poses dataset of tennis video sequences and we apply machine learning models to detect when a shot is made in the videos. Particularly we build a frame by frame human poses dataset of tennis games video sequences, using the most advanced human poses recognition tools. We label the shot frames, those where the player hits the ball with the racket. Then we apply specific normalization to center each frame compared to the player's body. Next we test several criteria to aggregate consecutive frames into groups, and in each group we compute distance based features to capture the movements of the player and their direction. For each combination, we apply several machine learning models that try to predict in every of those groups whether a shot is present. Results show us that boosting models are able to accurately capture the shot sequences in a video and to separate them from those where there is not a shot