Model-Based Stochastic Adaptive Search Algorithms for Global Optimization

We study a class of random sampling-based algorithms for solving general non-convex, nondifferentiable global optimization problems. These algorithms, collectively known as the model-based methods, are iterative approaches that successively estimate the optimal solution by sampling candidate solutions from a sequence of probability distribution models over the feasible region. We present several algorithm instantiations of model-based methods and discuss a systematic framework to investigate the convergence and asymptotic convergence rates of these algorithms by exploiting their connections to the well-known stochastic approximation (SA) method. Such an SA framework unifies our understanding of these randomized algorithms and provides new insight into their design and implementation issues. Our numerical experiments indicate that new implementations of these algorithms based on the proposed framework may lead to improved performance over existing procedures