The Containerisation of the ICON model for quasi-biennial oscillation simulations

ETH/CSCS has the duty of providing stable computing resources for its customers' applications. In climate research this requirement often requires stable, reproducible execution of a particular application over multiple years. Since the underlying computing platform – in our case CSCS Piz Daint – changes with each upgrade, ensuring such continuity for users is at best challenging. ESiWACE2 is putting a large effort to evaluate containers to see if they provide the platform stability needed. The object of this milestone, One model ported using containers, is to illustrate first-hand the viability of the container paradigm. The PRACE-supported QUBICC project which uses the Icosahedral Non-hydrostatic (ICON) model to simulate the evolution of the atmosphere in order to analyse the quasi-biennial oscillation (QBO) – the roughly 2-year shift from prevailing westerly mean flow to easterly and back – is one such application. The underlying high global resolution (2.9–5km) simulations require extensive computing resources over long periods of time, where the system needs to remain as static as possible. High performance is key to these simulations in order to complete them in reasonable time, so the overhead for running them inside a container has to be small. While QUBICC simulations currently run on 'bare-metal', i.e., not within containers, ETH/CSCS and MPI-M are preparing the container infrastructure to support these simulations over the long term where machine upgrades are unavoidable. The ICON model was one of many models addressed in the Container Hackathon for Modellers, a three-day event to 'jump start' the insertion of models into Docker-based containers. The results of this event are exhaustively documented in ESiWACE-2 Deliverable D2.8. Of all the applications ported to containers during the hackathon, the ICON team arguably made the least progress. The work was subsequently continued by the ETH/CSCS User Support team: Rafael Samiento (ETH/CSCS) methodically extended the initial hackathon work and upgraded to the code base on the QUBICC branch mc10_rrtmgp_remerge from the icon-cscs repository. The realisation can be found in a subsequent section. The port of the ICON for QUBICC using containers was thus a complete success as illustrated by the performance comparison (see Appendix A.1) with the base code without containers, using a medium resolution simulation on 64 to 160 GPU nodes of Piz Daint: The overhead of running with containers turns out to be minimal. The resulting container infrastructure is highly useful for CSCS customers who need ICON to remain in a stable configuration for periods much longer than the normal upgrade cycle. This milestone documents work contributing to Principle Objective (1) Enable leading European weather and climate models to leverage the available performance of pre-exascale systems with regard to both compute and data capacity in 2021. It enables the long simulations common in climate modelling on the rapidly changing environment of TIER0 supercomputers. By using Docker and Sarus, it makes use of the emerging container technology and thus contributes to the specific objective (b)Establish new technologies for weather and climate modelling"