Orchestra: rapid collaborative sharing of dynamic data

Conventional data integration techniques employ a “top-down” design philosophy, starting by assessing requirements and defining a global schema, and then mapping data sources to that schema. This works well if the problem domain is well-understood and relatively static, as with enterprise data. However, it is fundamentally mismatched with the “bottom-up” model of scientific data sharing, in which new data needs to be rapidly developed, published, and then assessed, filtered, and revised by others. We address the need for bottom-up collaborative data sharing, in which independent researchers or groups with different goals, schemas, and data can share information in the absence of global agreement. Each group independently curates, revises, and extends its data; eventually the groups compare and reconcile their changes, but they are not required to agree. This paper describes our initial design and prototype of the ORCHESTRA system, which focuses on managing disagreement among multiple data representations and instances. Our work represents an important evolution of the concepts of peer-to-peer data sharing [23], which considers revision, disagreement, authority, and intermittent participation