Rigorous analysis of some simple adaptive ES

Based on the theory of non-negative supermartingales, convergence results are proven for adaptive (1,)-ES with Gaussian mutations, and geometrical convergence rates are derived. In the d-dimensional case (d > 1), the algorithm studied here uses a different step-size update in each direction. However, the critical value for the step-size, and the resulting convergence rate do not depend on the dimension. Those results are discussed with respect to previous works. Thorough numerical investigations on some 1-dimensional functions validate the theoretical results