System for Intuitive Interaction With Large Scale Visual Databases.

In the recent several years, we have witnessed an astounding growth in the amount of visual media produced and consumed over the Internet. In this enormous digital landfill, user’s experience of utilising such a vast resource is rapidly degrading. This is mainly due to the problem of semantic gap, between the descriptions that can be extracted from the visual content and user’s expectations to access, search and interact with the visual content in a meaningful way. This thesis describes the design of a user-centred method for interaction with multimedia content, including retrieval and browsing, that departs from a typical paradigm of determining semantic descriptions of visual content. The proposed method takes advantage of user’s visual reasoning, and facilitates intuitive and interactive access to large-scale databases of visual content, achieving good results in the search context and excellent user experience. The thesis presents main contributions to research towards a intuitive interactive interface to a large-scale visual database, named FreeEye. In order to support different modalities of visual content, initial study was focused on video summarisation, achieving good results in fast and robust extraction of image based representation of video data. A novel frame ranking method is introduced subsequently. The experimental results demonstrated good subjective summarisation of the repetitive content, while maintaining notion of the temporal structure and knowledge discovery. Following the novel user-centric paradigm of large-scale image retrieval and interaction, the methodology and design of the FreeEye interface and supporting clustering engine create the core innovation of this research. The experimental results demonstrate that the system is very usable and intuitive, while offering pleasant browsing of visual data and often offering new perspectives of the same dataset by making surprising links between the data subsets. After a number of thorough user studies of the designed system in several application and contexts, from personal photography and interactive video search to film archives, the performance and usability of the FreeEye interface have proven to be exceptional. The tool in its simplicity and intuitiveness can be used as a general user interface for selecting media from a large collection, in spite of the main drawback in precision of the interactive search results