Video annotation by motion interpretation using optical flow streams

A new approach to automatic annotation of video sequences by dom-inant motion interpretation is presented. Unlike others, we separate the optical flow into two categories - singular and non-singular - which as we show is a more natural way of classif-ication for the purpose of dominant motion interpretation. We show that identification of patterns created by such natural categories, which can be observed from the measured optical flow, can help focus the interpretation of dominant motion in video segments. For robust observation of such natural patterns, we propose the computation of optical flow treams (OFS) from the video data and analyse the OFS for extraction of dominant motion content in the video segments. Our roposed approach has both bottom-up and top-down schemes suitably applied. The bottom-up scheme computes the OFS purely by local optimization of optical-flow equation. Then, the top-down scheme interprets the signature of the projection of the OFS on to the image coordinates for the detection of the natural category of the observed flow. Finally, further bottom-up analyses are done for sub-classification of the motion content in the video segments. The advantage of the pro-posed pproach is robustness in the extraction of dominant motion content in a video segment. We demonstrate this on a variety of real video sequences by generating the automatic motion annota-tion of the video frames and comparing with manual motion annota-tion