Towards a Visual Grep: A systematic analysis of various methods to compare video sequences

In this paper, we consider the problem of similarity between video sequences. Three basic questions are raised and (partially) answered. Firstly, at what temporal duration can video sequences be compared? The frame, shot, scene and video levels are identified. Secondly, given some image or video feature, what are the requirements on its distance measure and how can it be ?easily? transformed into the visual similarity desired by the inquirer? Thirdly, how can video sequences be compared at different levels? A general approach based on either a set or sequence representa-tion with variable degrees of aggregation is proposed and applied recursively over the different levels of temporal res-olution. It allows the inquirer to fully control the importance of temporal ordering and duration. Promising experimental results are presented