D8.2.1: Technical Framework for Evaluation of PCP Results

This deliverable provides a description of the organization, framework and items needed to perform the evaluation of the PRACE-3IP pre-commercial procurement (PCP) results. The responsibilities, tasks and overall process for performing the evaluation will be discussed in detail and the key elements involved in the implementation will be identified. The document will be updated as necessary during the progress of the PCP. The PCP partners have formed an entity called the Group of Procurers (GoP) to supervise the PCP. Within this group, committees, working groups and roles have been designated for different tasks within the PCP process. A three-fold organization is to be used: steering of the PCP by a GoP Committee, execution of the evaluations by an Assessment Committee together with work package 8 and task 2.1 of PRACE-3IP, and coordination of the work by a pair of coordinators. The PCP process has been organized into two major stages, the tendering stage using an “open procedure” and the execution stage which consists of three progressive phases where the candidates are evaluated. During the PCP, there are four evaluation milestones, one in the tendering stage and three at the end of each of the phases in the execution stage. For the evaluation of the pilot systems to be developed, a set of representative applications and synthetic benchmarks have been selected to assess the solutions with realistic workloads. In order to quantify the energy-efficiency of the solution in a reliable and consistent way, we have also developed a set of procedures and requirements for analyzing the energy consumption of each proposed system using a virtual data centre concept