Robust Canonical Time Warping for the Alignment of Grossly Corrupted Sequences

Temporal alignment of human behaviour from visual data is a very challenging problem due to a numerous rea-sons, including possible large temporal scale differences, inter/intra subject variability and, more importantly, due to the presence of gross errors and outliers. Gross errors are often in abundance due to incorrect localization and track-ing, presence of partial occlusion etc. Furthermore, such errors rarely follow a Gaussian distribution, which is the de-facto assumption in machine learning methods. In this paper, building on recent advances on rank minimization and compressive sensing, a novel, robust to gross errors temporal alignment method is proposed. While previous ap-proaches combine the dynamic time warping (DTW) with low-dimensional projections that maximally correlate two sequences, we aim to learn two underlying projection matri-ces (one for each sequence), which not only maximally cor-relate the sequences but, at the same time, efficiently remove the possible corruptions in any datum in the sequences. The projections are obtained by minimizing the weighted sum of nuclear and 1 norms, by solving a sequence of convex op-timization problems, while the temporal alignment is found by applying the DTW in an alternating fashion. The supe-riority of the proposed method against the state-of-the-art time alignment methods, namely the canonical time warp-ing and the generalized time warping, is indicated by the experimental results on both synthetic and real datasets. 1