A Fully Automated Content Based Video Search Engine Supporting Spatio-Temporal Queries

The rapidity with which digital information, particularly video, is being generated, has necessitated the development of tools for efficient search of these media. Content based visual queries have been primarily focused on still image retrieval. In this paper, we propose a novel, interactive system on the Web, based on the visual paradigm, with spatio-temporal attributes playing a key role in video retrieval. We have developed innovative algorithms for automated video object segmentation and tracking and use real-time video editing techniques while responding to user queries. The resulting system called VideoQ 1 is the first on-line video search engine supporting automatic object based indexing and spatio-temporal queries. The system performs well, with the user being able to retrieve complex video clips such as those of skiers, baseball players, with ease. 1 Demo available at http://www.ctr.columbia.edu/VideoQ/; A shorter version of this paper appeared at the ACM Conference on Mul..