Information Retrieval, Information Structure, and Information Agents

This paper presents a customizable architecture for software agents that capture and access information in large, heterogeneous, distributed electronic repositories. The key idea is to exploit underlying structure at various levels of granularity to build high-level indices with task-specific interpretations. Information agents construct such indices and are configured as a network of reusable modules called structure detectors and segmenters. We illustrate our architecture with the design and implementation of smart information filters in two contexts: retrieving stock market data from Internet newsgroups, and retrieving technical reports from Internet ftp sites. 1 Introduction The proliferation of information in electronic form and the development of high-speed networking make the problem of locating and retrieving information in vast electronic environments one of the grand challenges of computer science. Examples of electronic corpora include repositories of newspapers and technic..