Adaptive Policies for Stochastic Systems under a Randomized Discounted Cost Criterion∗

The paper deals with a class of discrete-time stochastic control pro-cesses under a discounted optimality criterion with random discount rate, and possibly unbounded costs. The state process {xt} and the discount process {αt} evolve according to the coupled difference equa-tions xt+1 = F (xt, αt, at, ξt), αt+1 = G(αt, ηt) where the state and discount disturbance processes {ξt} and {ηt} are sequences of i.i.d. random variables with unknown distributions θξ and θη respectively. Assuming observability of the process {(ξt, ηt)} , we use the empirical estimator of its distribution to construct asymptotically discounted optimal policies. Key Words: Empirical distribution; discrete-time stochastic sys-tems; discounted cost criterion; random rate; optimal adaptive policy