DISCRETIZED MAXIMUM LIKELIHOOD AND ALMOST OPTIMAL ADAPTIVE CONTROL OF ERGODIC MARKOV MODELS

Abstract. Three distinct controlled ergodic Markov models are considered here. The models are a discrete time controlled Markov process with complete observations, a controlled diusion process with complete observations, and a discrete time controlled Markov process with partial observations. The partial observations for the third model have the special form of complete observations in a xed recurrent set and noisy observations in its complement. For each of the models an almost self-optimizing adaptive control is given. These adaptive controls are constructed from a family of estimates that use a nite discretization of the parameter set and a nite family of almost optimal ergodic controls by a randomized certainty equivalence method. A continuity property of the infor-mation of a model for one parameter value with respect to another is used to establish this almost optimality property