Querying Heterogeneous Information Sources Using Source Descriptions

We witness a rapid increase in the number of structured information sources that are available online, especially on the WWW. These sources store interrelated data on topics such as product information, stock market information, entertainment, etc. We would like to use the data stored in these databases to answer complex queries that go beyond keyword searches. We describe the Information Manifold, an implemented system that provides uniform access to a heterogeneous collection of more than 100 information sources, on the WWW. IM contains declarative descriptions of the contents and capabilities of the information sources. We describe algorithms that use the source descriptions to prune efficiently the set of information sources for a given query and practical algorithms to generate executable query plans. We also present experimental studies that indicate that the architecture and algorithms used in the Information Manifold scale up well to several hundred information sources