Damaris: In Situ Data Analysis and Visualization in Support of Large-Scale CFD Simulations

International audienceNowadays, extreme-scale simulations are widely adopted in the study of modern computational fluid dynamic (CFD) problems. In the traditional approach, datasets resulting from the simulation are typically shipped to some auxiliary post-processing platforms for offline visualization and further analysis, that is very costly in terms of storage requirements and performance impacts. In addition, no scientifically usable result is available before the end of the post-processing phase. To resolve these issues, the high performance computing (HPC) community has shown a considerable interest in "in situ" processing, that is, visualization and analysis of simulation data while the simulation is still running. The most important reasons in favor of in situ processing are: 1) it helps reducing the I/O costs by not writing the results first on disk and then loading them back in memory; 2) it provides the potential to use all available resources (e.g. GPUs) in the supercomputer that runs the simulation; 3) it provides support for computational steering of the simulation by allowing users to change the input configuration at run time. In this presentation, we introduce Damaris, a middleware for in situ data analysis and visualization targeting extreme-scale, MPI-based simulations. The main goal of Damaris is to provide a simple method to instrument a simulation in order to benefit from in situ analysis and visualization. To this aim, the computing resources are partitioned such that a subset of cores in a SMP node or a subset of nodes of the underlying platform are dedicated to in situ processing. The data generated by the simulation are passed to these dedicated processes either through shared memory (in the case of dedicated cores) or through the MPI calls (in the case of dedicated nodes) and can be processed both in synchronous and asynchronous modes. Afterwards, the processed data can be visualized (using VisIt or ParaView connectors) or dumped (in HDF5 format) using out-of-the-box Damaris plug-ins. It is also possible to develop custom plug-ins to support any other in situ processing steps such as feature extraction or data compression and run them in dedicated processes. Damaris also supports a very simple API to instrument simulations. Moreover, using some XML conguration les for dening simulation data types (e.g. meshes) makes the instrumentation process easier with minimum code modications.Finally, we report the results of some experiments we made to evaluate Damaris with respect to scalability and performance. These experiments were conducted on BlueWaters (on 6,400 cores) and Grid'5000 (on 960 cores). In these experiments Damaris has been employed to visualize the data generated by the CM1 atmospheric simulation, the CROCO coastal and ocean simulation, and the Nek5000 CFD solver. During the experiments, the impact of Damaris is measured by comparing the simulations instrumented by Damaris (space partitioning approach) with a baseline where those simulations include data processing codes directly on their source code (time partitioning approach). The results of these simulations show that the incorporation of Damaris into a simulation decreases the total run time of the simulation due to its asynchronous data processing capabilities. In addition, using Damaris for data visualization has nearly no impact on the total run time of a simulation. We also report that the scalability of the simulations that benet from Damaris is nearly linear compared with the ideal scalability of a large-scale simulation