Creation of a database warehouse infrastructure for storing and organizing data for online performance assessment experiments

Appropriate benchmark design is a requirement for assessing and improving bioinformatics methods and tools, and needs of best practices to pass over from raw data to valuable knowledge for decision-making. In order to build a continuous automated benchmarking infrastructure which hosts different - emerging or existing - benchmark efforts, bioinformatics tools and data-types, it is crucial to discuss storage, analysis, comparison and sharing of large heterogeneous data sets. Finally, the most appropriate format to communicate results by either exposing them to third parties resources e.g. tools registries, and/or directly via a web-portal. The increasing complexity of the constantly growing body of biological data e.g. unstructured description of resources, non-standardized input and output formats, lack of appropriate metadata and/or deprecated software source codes, represents a tremendous challenge. Thus, we must develop new technical solutions or adapt existing ones to leverage existing data and to better prepare for future challenges around the scientific, technical and functional evaluation of bioinformatics methods and tools. Deliverable 2.1. defines the data warehouse infrastructure needed to host different benchmark initiatives from a broad range of bioinformatics fields, provides a reference implementation, and oversee the overall ELIXIR Tools and Data Services Registry integration and operation. On this report we will examine the following aspects of a data warehouse: Background about the need of an automated benchmark infrastructure. An architecture overview with an emphasis in the data warehouse as central component. An explanation about the database warehouse design and implementation. A final review about future works in the context of the benchmark infrastructure