Optimal Controller Synthesis for Nonlinear Dynamical Systems

Abstract — This work presents a novel method for synthesiz-ing optimal Control Lyapunov functions for nonlinear, stochas-tic systems. The technique relies on solutions to the linear Hamilton Jacobi Bellman (HJB) equation, a transformation of the classical nonlinear HJB partial differential equation to a linear partial differential equation, possible when a particular structural constraint on the stochastic forcing on the system is satisfied. The linear partial differential equation is viewed as a set of constraints which are in turn relaxed to a linear differ-ential inclusion. This allows for the optimization of a candidate polynomial solution using sum of squares programming. The resulting polynomials are in fact viscosity solutions of the HJB, allowing for the well developed results in the theory of viscosity solutions to be applied to these numerically generated solutions. It is shown how the viscosity solutions may be made arbitrarily close to the optimal solution via a hierarchy of semidefinite optimization problems. Furthermore, this work develops a-priori bounds on trajectory suboptimality when using these approximate value functions. I