Neural Networks for Nonlinear Dynamical Systems

Thesis (Ph.D.)--University of Washington, 2022Nonlinear dynamical systems are ubiquitous in many fields of sciences and engineering. Throughout the history, differential equations were widely accepted as effective tools for describing the evolution of such systems in continuous time, with the earliest example dating back to the mid-1600s when Isaac Newton discovered his laws of motion and universal gravitation, and combined them to explain the planetary motion. However, two difficulties often rise in practice. First, identifying governing law of a system requires tremendous insights. The dynamics are often so complex that even to build up a coarse-grained model is a nontrivial task. Second, obtaining solutions from the governing equation is also challenging, most of which need to resort to numerical simulations. And high-fidelity simulations are often expensive to conduct. Thanks to the recent advances in deep learning and the abundance of measurement data, now we have alternative ways to address these two challenges. In this thesis, we present several neural network architectures for nonlinear dynamical systems. We begin by introducing neural network-based hierarchical time-steppers (HiTS) which enables highly accurate and efficient numerical integration over a long period of time. Then we turn to multi-resolution convolutional autoencoder (MrCAE) which integrates and leverages three highly successful mathematical architectures: (i) multigrid methods, (ii) convolutional autoencoders and (iii) transfer learning, providing an adaptive, hierarchical architecture that capitalizes on a progressive training approach for multiscale spatio-temporal data. Finally, we introduce physics-informed Koopman network (PIKN) which leverages automatic differentiation to impose the underlying physical laws via soft penalty constraints so that to identify the intrinsic, linearized coordinates for nonlinear dynamics. The first two architectures are purely data-driven, aiming to provide an alternative way to model nonlinear dynamical systems and run efficient simulations. The third architecture is physics-informed, providing a data-efficient framework to numerically approximate the Koopman operator, enabling fast simulations