A study on metamodeling techniques, ensembles, and multi-surrogates in evolutionary computation

Surrogate-Assisted Memetic Algorithm (SAMA) is a hybrid evolutionary algorithm, particularly a memetic algorithm that employs surrogate models in the optimization search. Since most of the objective function evaluations in SAMA are approximated, the search performance of SAMA is likely to be affected by the characteristics of the models used. In this paper, we study the search performance of using different meta modeling techniques, ensembles, and multi-surrogates in SAMA. In particular, we consider the SAMA-TRF, a SAMA model management framework that incorporates a trust region scheme for interleaving use of exact objective function with computationally cheap local meta models during local searches. Four different metamodels, namely Gaussian Process (GP), Radial Basis Function (RBF), Polynomial Regression (PR), and Extreme Learning Machine (ELM) neural network are used in the study. Empirical results obtained show that while some metamodeling techniques perform best on particular benchmark problems, ensemble of metamodels and multisurrogates yield robust and improved solution quality on the benchmark problems in general, for the same computational budget