Table of Contents

Abstract. We describe two complementary methods for obtaining the principal objects, characters and scenes in a video. In one we use the temporal editing structure of the shots to cluster the video into parts that play in the same physical location. In the other method we ‘discover ’ the principal objects, characters and scenes by measuring the re-occurrence of spatial configurations of viewpoint invariant features. The video can then be annotated with these objects, locations etc. The problem is challenging firstly because an object can undergo substan-tial changes in imaged appearance throughout a video (due to viewpoint and illumination changes, and partial occlusion), and secondly because configurations are detected imperfectly, so that inexact patterns must be matched. The novelty of the method is that viewpoint invariant features are used, and that (in the second method) efficient methods from the text analysis literature are employed to reduce the matching complexity. Examples of the scenes and discovered objects (at various spatial scales) are shown for several episodes of the situation comedy ‘Fawlty Towers’