Improving experimental design for automatic algorithm selection techniques by using a virtual best solver

Evidence of the quality of metaheuristics is usually empirical. Common experimental design consists of testing a technique on benchmark instances. The goal of this article is first, to expose the flaws of this approach. Then, to illustrate how the experimental design can be improved by calculating a performance upper bound based on the instances and algorithms used. And ultimately, to introduce a tighter upper bound that also takes information about the problem features into account.status: publishe