Bayesian and Minimax Solutions to the Adaptation of a Plant to a Changing Environment

Abstract In this paper, we consider the decision problem that an automated control system faces when its plant has to adapt to a changing environment which can be in a finite number of states. The law of transition between states is unknown, and we hence adopt the worst case scenario. The controller gathers data at regular intervals and, periodically, it has to select the best control for the next period. The optimality criterion used in the selection process is the minimum expected loss in utility from not applying the best control. We construct a model and propose solution procedures for both stable and unstable environments. It is shown that when the environment is stable, the decision problem is equivalent to a classification problem. When it is unstable, we formulate the problem as a two-person zero-sum game. In both cases, the minimax principle plays a major role