D7.4: Evaluation of Tools and Techniques for Future Exascale Systems

The objective of the PRACE-4IP Work Package 7 (WP7) ‘Application Enabling and Support’ is to provide enabling support for High Performance Computing (HPC) applications codes, to ensure that these applications can effectively exploit multi-petaflop systems and future PRACE Exascale systems. The Task 7.2a, within WP7, ‘Preparing for Future PRACE Exascale Systems’ aims to investigate the various programming tools, languages, libraries and algorithms needed for future Exascale systems through an analysis and exploitation phase. In this deliverable, we report on the exploitation of state-of-the-art HPC tools and techniques on different codes that are of interest to the European scientific and engineering research community, and where possible, with a focus on European Centres of Excellence (CoEs) interests and requirements. In this sense, the report here follows on naturally from deliverable D7.3, ‘Inventory of Exascale Tools and Techniques’, which represented the first phase of activity in T7.2a. Much of the exploitation work reported on here, was inspired by the comprehensive and in-depth analysis of European CoE requirements and state-of-the-art HPC tools and techniques as reported in D7.3 (as well as D7.2.1 in PRACE-3IP). We focus on five separate topics that we have identified as being important to enable European applications on the road to exascale, and which mirror four of the topics reported on in the survey of D7.3. These fours topics are: Programming Models, Scalable Libraries and Algorithms, Debuggers and Profilers and I/O Management Techniques. The fifth topic we have chosen to focus on is Energy Efficiency, which is becoming an increasingly relevant consideration on deep-petascale systems, and will become an even greater challenge to contend with on future European extreme-scale systems. In particular, we see the need for an increased focus on energy efficient computing within WP7 in PRACE-5IP in order to take full advantage of the solutions being delivered as part of the final phase of the PRACE Pre-commercial Procurement (PCP) and beyond. While some of the tools and techiques we have exploited are well established and standardized, such as MPI and OpenMP, we have also driven effort into exploring newer, pathbreaking tools, such as the newly developed (and implemented) cache-aware roofline model, novel adaptive sparse solvers, high-resolution energy measurement systems and a prototype runtime-tuning tool for increased energy efficiency at exteme-scale, the latter emerging from one of the FET-HPC projects. We feel strongly that the engagement between PRACE and the FET-HPC projects (as well as other Exascale research and development projects) reflected by such WP7 projects should be further expanded and strengthened during PRACE-5IP. Below, we provide a flavour of some of the investigative work carried out as part of the T7.2a exploitation phase. For a more detailed description of each of the projects summarised in this document, we refer the reader to the PRACE-4IP whitepapers associated with each of the 15 projects. Energy Efficient Computing Throughout the lifetime of PRACE, the energy consumption of the large-scale European HPC systems has continued to increase, which poses a significant challenge in meeting the power constraints placed on any future European Exascale system by the early 2020s time-frame. While there are many ongoing initiatives to confront the challenge of energy consumption on the hardware level, it is now increasingly appreciated that a more holistic approach is required that includes all layers of the “HPC stack”, including the application layer. Several of the CoEs are keenly focused on improving the energy efficiency of their applications running on extreme-scale systems, including MaX, ESiWACE and PoP (as well as the ESCAPE FET-HPC project) and it is with their interests in mind, that we investigate a prototype tool suite, known as READEX, being developed by the H2020 FET-HPC project (also named READEX). We find that a pre-alpha version of the READEX tool suite can already identify dynamism in highly scalable European applications that regularly use large-scale PRACE resources, which allows for the determination of their tuning potential for improving energy efficiency on such systems. We also apply the prototype tool to a proxy version of the well-known molecular dynamics code, NAMD, which is of interest to the ECAM and MaX CoEs. Programming Interfaces and Standards We note from the survey of CoE requirements carried during the first phase of T7.2a, that many of the CoEs have not yet shown evidence of exploring emerging or novel programming models, beyond the now well-established models, such as MPI, OpenMP, CUDA and OpenCL. As a result, we have focused mainly on exploiting these well known models on emerging many-core systems and, where appropriate have focused on investigating typically underutilized modern features of these standards (e.g., MPI 3.0). We see the lack of awareness of emerging and novel programming models, as well as a possible aversion to risk in utilizing these, as a worthwhile challenge for WP7 to confront more aggressively beyond PRACE 4IP. With a view to future exploitation of programming models (e.g. in PRACE 5IP), we feel that there is an important role for WP7 to play in carrying out exploratory work on more novel prototype-level programming models and tools, as well as to increase awareness of such programming models within the CoEs. Debuggers and Profilers Improving energy-to-solution and time-to-solution in European weather forecasting codes is a key objective of the ESiWACE CoE and even more so for the H2020 FET-HPC ESCAPE project. Such focus includes targeting the IFS weather code by a ‘divide-and-conquer’ approach in optimizing its most heavily used subroutines through the development of so-called “Weather Dwarfs”. This goal is predicated on a move to European Exascale computing, which is expected to exploit massively parallel node-level chip architectures, a “stepping-stone” example of which is the recently released Intel Xeon Phi Knights Landing (KNL) processor. With these challenges in mind, we investigate the powerful potential of the recently developed cache-aware roofline model (CARM) profiling technique to better understand where bottlenecks exist and improve performance for so-called “European Weather Dwarfs” on the Intel Xeon Phi platform. Scalable Libraries and Algorithms Several of the 23 application codes within the EoCoE CoE are concerned with the solution of large sparse linear systems, where the performance of such solvers greatly influences the overall scalability of the applications. We have focused on improving the scalability of two heavily used application codes within EoCoE (TRACES and HORSE), by investigating the potential of new algebraic hybrid iterative/direct solvers within these codes as an alternative to algebraic multigrid preconditioned Krylov iterative solvers and preconditioners, which are known to scale poorly at large scale. We also describe two applications for illustrating the modeling capabilities of the simulation software supported by EoCoE and assessing their parallel performance on the road to Exascale: the scattering of a plane wave by an aircraft, and the interaction of an electromagnetic wave with a heterogeneous model of head tissues. I/O Management Techniques Parallel I/O performance plays a key role in many high-performance computing (HPC) applications. I/O bottlenecks are an important challenge to understand and, where possible, eliminate on both current petascale systems and looking forward to exascale computing. It is therefore necessary for research communities with high I/O requirements to understand the parallel I/O performance of existing HPC systems and applications to be suitably equipped to make informed plans for future procurements and software development projects. With this challenge in mind, in this deliverable we report on how we have worked closely with the ESiWACE CoE to better understand its I/O requirements and have focused on developing and running I/O benchmarks to provide deeper insight into performance across a range of large-scale European HPC systems. Given the ubiquity of I/O challenges across HPC domains, the findings should be of interest to the majority of the other CoEs