A rule-based approach to represent spatio-temporal relations in video data

Abstract. In this paper, we propose a new approach for high level seg-mentation of a video clip into shots using spatio-temporal relationships between objects in video frames. The technique we use is simple, yet novel and powerful in terms of eectiveness and user query satisfaction. Video clips are segmented into shots whenever the current set of relations between objects changes and the video frames where these changes have occurred are chosen as key frames. The topological and directional relati-ons used for shots are those of the key frames that have been selected to represent shots and this information is kept, along with key frame inter-vals, in a knowledge-base as Prolog facts. We also have a comprehensive set of inference rules in order to reduce the number of facts stored in our knowledge-base because a considerable number of facts, which otherwise would have to be stored explicitly, can be derived by these rules with some extra eort