A stochastic gradient type algorithm for closed loop problems

We focus on solving closed-loop stochastic problems, and propose a perturbed gradient algorithm to achieve this goal. The main hurdle in such problems is the fact that the control variables are in¯nite dimensional, and have hence to be represented in a ¯nite way in order to numerically solve the problem. In the same way, the gradient of the criterion is itself an in¯nite dimensional object. Our algorithm replaces this exact (and unknown) gradient by a per-turbed one, which consists in the product of the true gradient evaluated at a random point and a kernel function which extends this gradient to the neigh-bourhood of the random point. Proceeding this way, we explore the whole space iteration after iteration through random points. Since each kernel func-tion is perfectly known by a ¯nite (and small) number of parameters, say N, the control at iteration k is perfectly known as an in¯nite dimensional object by at most N £ k parameters. The main strength of this method is that it avoids any discretization of the underlying space, provided that we can draw as many points as needed in this space. Hence, we can take into account in a new way the possible measura-bility constraints of the problem. Moreover, the randomization of this algorithm implies that the most probable parts of the space are the most explored ones, what is a priori an interesting feature. In this paper, we ¯rst show a convergence result of this algorithm in the gen-eral case, and then give a few numerical examples showing the interest of this method for solving practical stochastic optimization problems