Automata theory meets approximate dynamic programming: Optimal control with temporal logic constraints

We investigate the synthesis of optimal controllers for continuous-time and continuous-state systems under temporal logic specifications. The specification is expressed as a deterministic, finite automaton (the specification automaton) with transition costs, and the optimal system behavior is captured by a cost function that is integrated over time. We construct a dynamic programming problem over the product of the underlying continuous-time, continuous-state system and the discrete specification automaton. To solve this dynamic program, we propose controller synthesis algorithms based on approximate dynamic programming (ADP) for both linear and nonlinear systems under temporal logic constraints. We argue that ADP allows treating the synthesis problem directly, without forming expensive discrete abstractions. We show that, for linear systems under co-safe temporal logic constraints, the ADP solution reduces to a single semidefinite program