Automated Generation of Benchmarks with High Discriminatory Power for Specific Sets of Black Box Search Algorithms

Black box search algorithms (BBSAs) vary widely in their effectiveness at solving particular classes of problems. It is paramount to be able to identify the most effective BBSA for a real-world problem. There exists a sizable set of standard benchmarks, but only for very narrow problem classes. Any such limited set is inherently unable to identify the general differences in effectiveness between particular BBSAs and practically infeasible for a sufficiently comprehensive comparison to differentiate between arbitrary BBSAs. Thus it is necessary to create custom benchmarks for given BBSAs to obtain benchmarks with high discriminatory power, utilizing a meta evolutionary algorithm approach in this case. By sampling the search space of a realworld problem and identifying the most similar custom benchmark, the previously identified BBSA with the best performance may be expected to be the most effective for solving that real-world problem. Results are presented showing the effectiveness of this approach