Recent Approaches to the Control of Stochastic Processes

Stochastic Control Theory is concerned with the control of dynamical systems which are random in some sense. In recent years the description of systems by first-order vector differential or difference equations has given a big impetus to control theory. It is the purpose of this paper to give a simple introduction to the theory of optimal control of such systems when random disturbances are present. Following the definition of a few terms, a simple discrete example, due to Dreyfus, is discussed to illustrate the main con-cepts. The dynamic programming procedure is then applied to discrete-time and continuous-time completely observable systems. Discrete-time partially observable systems are then discussed. The extension to continuous-time systems raises many interesting problems (e.g. the evolution of conditional probability densities and the whole subject of non-linear filtering) but is beyond the scope of an introductory paper. Simple examples are given to illustrate many points. 1