Stochastic Control with Uncertainty in the Parameters through Dynamic Programming in the Belief Space

We consider some problems of optimal control with discrete time where some parameters are fixed but unknown. We translate the problem into one of stochastic control in the belief state, and compare some approximations to the solution. As we observe the system evolution, our uncertainty over the unknown parameters is updated. We show applications in which it is necessary to make trade-offs between risk, return and learning. We evaluate the impact of prior information