Automatic Differentiation and Scientific Machine Learning (SciML) in Reality: what can go wrong, and what to do about it.

Scientific machine learning (SciML) through differentiable simulation is a method that is faster and more robust than techniques like physics-informed neural networks (PINNs), neural operators, and other techniques. However, doing differentiable simulation correctly requires a deep understanding of automatic differentiation and the numerical properties of simulation. In this talk, we dive deep into the numerical stability of derivatives of simulation processes, showing how naive applications of differentiable programming can give surprisingly incorrect results, and how one may need to modify simulations in order to perform robust automated model discovery and calibration.Originally part of the JuliaHEP 2023 Workshop at the ECAP (Erlangen Centre for Astroparticle Physics), https://indico.cern.ch/event/1292759/Bio: Bio: Dr. Chris Rackauckas is the VP of Modeling and Simulation at JuliaHub, the Director of Scientific Research at Pumas-AI, Co-PI of the Julia Lab at MIT, and the lead developer of the SciML Open Source Software Organization. For his work in mechanistic machine learning, his work is credited for the 15,000x acceleration of NASA Launch Services simulations and recently demonstrated a 60x-570x acceleration over Modelica tools in HVAC simulation, earning Chris the US Air Force Artificial Intelligence Accelerator Scientific Excellence Award. See more at https://chrisrackauckas.com/. He is the lead developer of the Pumas project and has received a top presentation award at every ACoP in the last 3 years for improving methods for uncertainty quantification, automated GPU acceleration of nonlinear mixed effects modeling (NLME), and machine learning assisted construction of NLME models with DeepNLME. For these achievements, Chris received the Emerging Scientist award from ISoP.