Automatic generation of warehouse mediators using an ontology engine

The Data Foundry research project at LLNL is investigating data warehousing in highly dynamic scientific environments. Specifically, we are developing a data warehouse to aid structural biologists in genetics research. Upon completion, this warehouse will present a uniform view of data obtained from several heterogeneous data sources containing distinct but related data from various genetics domains. Our warehouse uses a mediated data warehouse architecture in which only some data is represented explicitly in the warehouse; remote access is required to obtain the non-materialized data. Mediators are used to convert data from the data source representation to the warehouse representation and make it available to the warehouse. The major challenge we face is reducing the impact of source schema changes on warehouse availability and reliability: based upon previous efforts, we anticipate one source schema modification every 2-4 weeks once all of the desired sources have been integrated. Incorporating these modifications into the mediators using brute force results in an unacceptable amount of warehouse down-time. We believe that extensive use of a carefully designed ontology will allow us to overcome this problem, while providing a useful knowledge base for other applications. In addition to automatically generating the transformation between the data sources and the warehouse, the ontology will be used to guide automatic schema evolution, and provide a high level interface to the warehouse. This paper focuses on the use of the ontology to automatically generate mediators, because reducing the effect of source changes is a critical step in providing reliable access to heterogeneous data sources