A Deep Learning Framework for Optimal Feedback Control of High-Dimensional Nonlinear Systems

Designing optimal feedback controllers for nonlinear dynamical systems requires solving Hamilton-Jacobi-Bellman equations, which are notoriously difficult when the state dimension is large. Existing strategies for optimal feedback design are usually not valid for high-dimensional problems, may rely on restrictive problem structures, or are valid only locally around some nominal trajectory. On the other hand, mature numerical methods exist for solving open loop optimal control problems, and these have been successfully deployed in a number of settings including the International Space Station. It is well-known, however, that open loop controls are not robust to model uncertainty or disturbances, so for real-time applications we need a closed loop feedback controller.In this dissertation we develop a deep learning-based framework to synthesize optimal feedback controllers for high-dimensional nonlinear systems. The core idea is to train a neural network feedback controller on data generated by solving a set of open loop optimal control problems, which does not require state space discretization and is thus applicable in high dimensions. We also introduce several specialized neural network architectures which guarantee local stability by construction and can still accurately approximate the optimal control over large domains. Training is made more effective and data-efficient by leveraging the known physics of the problem and using the partially-trained neural network to aid in adaptive data generation. Data generation and model training, while computationally demanding, are performed offline. Once trained, the neural network can be evaluated online at minimal cost, thus delivering real-time optimal feedback control. We demonstrate the feasibility and effectiveness of the proposed control design framework on several nonlinear high-dimensional examples, including stabilization of unstable Burgers'-like partial differential equations, attitude control of a rigid body satellite with momentum wheels, and altitude and course tracking for an unmanned aircraft