The Design of Discovery Net: Towards Open Grid Services for Knowledge Discovery

With the emergence of distributed resources and grid technologies there is a need to provide higher level in-formatics infrastructures allowing scientists to easily cre-ate and execute meaningful data integration and analysis processes that take advantage of the distributed nature of the available resources. These resources typically include heterogeneous data sources, computational resources for task execution and various application-specific services. The effort of the high performance community has so far mainly focused on the delivery of low-level informatics infrastructures enabling the basic needs of grid applica-tions. Such infrastructures are essential but do not di-rectly help end-users in creating generic and re-usable applications. In this paper, we present the Discovery Net architec-ture for building grid-based knowledge discovery applica-tions. Our architecture enables the creation of high-level, re-usable and distributed application workflows that use a variety of common types of distributed resources. It is built on top of standard protocols and standard infra-structures such as Globus but also defines its own proto-cols such as the Discovery Process Mark-up Language for data flow management. We discuss an implementa-tion of our architecture and evaluate it by building a real-time genome annotation environment on top