Fast Incremental Indexing for Full-Text Information Retrieval

Full-text information retrieval systems have tradi-tionally been designed for archival environments. They often provide little or no support for adding new documents to an existing document collec-tion, requiring instead that the entire collection be re-indexed. Modern applications, such as infor-mation filtering, operate in dynamic environments that require frequent additions to document collec-tions. We provide this ability using a traditional inverted file index built on top of a persistent ob-ject store. The data management facilities of the persistent object store are used to produce effi-cient incremental update of the inverted lists. We describe our system and present experimental re-sults showing superior incremental indexing and competitive query processing performance