Nonparametric adaptive control of discrete-time partially observable stochastic systems

AbstractWe consider discrete-time, partially observable stochastic control systems xt + 1 = F(xt, ut, vt), t = 0,1,…,yt = G(ut − 1, xt, wt), t = 1,2,…, y0 = G0(x0, w0), where xt, ut, and yt are, respectively, the state, control, and observation at time t, and the state and observation disturbance processes {vt} and {wt} are sequences of i.i.d. random elements with distributions ν and μ, respectively. We study the non-parametric adaptive control problem of maximizing the infinite-horizon discounted reward criterion when ν and μ are unknown and determine conditions for several adaptive policies to be asymptotically optimal and for almost-surely uniform approximations of the optimal reward function. Our approach combines convergence results for empirical processes and recent results on parameter-adaptive stochastic control problems