Control Theory: Course Summary

There are a wide range of problems which involve making decisions over time in the face of uncertainty. Control theory draws from the fields of Engineering, Operations research and Mathematics to supply tools to regulate or optimise a system. I will briefly introduce and discuss some relevant topics considered as part of control theory. Phenomena can be modelled as dynamical systems, where there is some level of control that can be imposed on the system. This includes process control, optimal stopping, automated planning and general strategic decision making in the face of uncertainty. In these problems there exists some optimal behaviour or optimal policy that is desired. Markov Decision Problems An important class of characterising problems are Markov decision problems, which are equivalently known as Markov decision processes (MDPs). Essentially most real-life stochas-tic control problems can be posed as an MDP. This class of problems is backed by an elegant theoretical framework and is underpinned by Bellman’s optimality equation (1), which can be used to solve Markov decision processes