A visual analysis of articulated motion complexity based on optical flow and spatial-temporal features

The understanding of motion is an important problem in computer vision with applications including crowd-flow analysis, video surveillance, and estimating three-dimensional structure. A less-explored problem is the visual characterization and quantification of motion complexity. An important motion class that is prevalent in living beings is articulated motion (segments connected by joints). At present, no known standardized measure for quantifying the complexity of articulated motion exists. Such a measure could facilitate advanced motion analysis with applications including video indexing, motion comparison, and advanced biological study of visual signals in organisms. This dissertation presents an in-depth study of the development of several complexity measures for visual articulated motion. Optical flow is the basis of many motion estimation approaches and our first measure utilizes this as the starting point. Using optical flow, we develop a set of features to characterize different aspects of the motion and combine them to estimate the complexity of the movement. The second measure also utilizes optical flow, but uses higher-order features as motion descriptors. Specifically, features that encode the periodic nature of movements, synchrony, and movement clusters are developed and used toward the design of a new and improved complexity measure. To validate the measure, a human study was conducted. Subjects were asked to (a) give motion complexity scores to a set of videos and (b) rank features based on their importance to complexity. Using this study, we developed prediction models to estimate the motion complexity and also classification models to classify the videos. We use an alternative approach for our third measure based on interesting motion points in the combined space-time domain. These spatial-temporal interest points integrate hidden complexity information in the movement sequence. High level features are proposed to capture different dimensions of movement complexity from these interest points and then combined to estimate the overall complexity of the movement. All three approaches have been evaluated using two datasets: human movements and wolf spider movements. Extensive evaluation of the measures show the accuracy of estimating the complexity of articulated motion, and demonstrate the efficacy of their use toward classifying motion based on complexity