openPMD - An Open Standard for Particle-Mesh Data Files

Data fuels and substantiates scientific discoveries. Advanced particle accelerator research is no different and has an inherent need for high-rate, high-resolution data. But in recent years, generating and handling the sheer amount data that is driving our discoveries became challenging. Just to name a few: scalable output from 3D simulations breaks down for modern supercomputers, high-rate Mpixel cameras generate GByte/s for laser control and comparing even just simulations to each other is a significant, manual, error-prone process. We present our open standard for particle and mesh based data, addressing these and more common challenges in our community. Based on state-of-the-art file formats and I/O libraries, we implement and improve scalable I/O without loosing self-description. openPMD is portable, truly self-describing, documented, forward-updatable and makes data comparable and reproducible. openPMD tries to follow best-practices towards an open-science workflow. While the meta standard is developed in an open, reviewable, versioned technical document, a large collection of tools and bindings develop around it. We will take a look at the community that fuels openPMD, the open-source projects evolving around it, adopters in the domain of astro-physics, photon-science and classical accelerator physics and the latest updates arriving in openPMD 2.0.0 .This research used resources of the Oak Ridge Leadership Computing Facility located in the Oak Ridge National Laboratory, which is supported by the Office of Science of the Department of Energy under Contract DE-AC05-00OR22725. This work was supported in part by the Director, Office of Science, Office of High Energy Physics, of the U.S. Department of Energy under Contract No. DE-AC0205CH11231, as part of the activities of the Consortium for Advanced Modeling of Particle Accelerators (CAMPA). This work was performed in part under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory (LLNL) under Contract DE-AC52-07NA27344