Natural evolution strategies

Editor: Una-May O’Reilly This paper presents Natural Evolution Strategies (NES), a recent family of black-box opti-mization algorithms that use the natural gradient to update a parameterized search distri-bution in the direction of higher expected fitness. We introduce a collection of techniques that address issues of convergence, robustness, sample complexity, computational complex-ity and sensitivity to hyperparameters. This paper explores a number of implementations of the NES family, such as general-purpose multi-variate normal distributions and separa-ble distributions tailored towards search in high dimensional spaces. Experimental results show best published performance on various standard benchmarks, as well as competitive performance on others