Benchmark of Bayesian Optimization and Metaheuristics for Control Engineering Tuning Problems with Crash Constraints

Controller tuning based on black-box optimization allows to automatically tune performance-critical parameters w.r.t. mostly arbitrary high-level closed-loop control objectives. However, a comprehensive benchmark of different black-box optimizers for control engineering problems has not yet been conducted. Therefore, in this contribution, 11 different versions of Bayesian optimization (BO) are compared with seven metaheuristics and other baselines on a set of ten deterministic simulative single-objective tuning problems in control. Results indicate that deterministic noise, low multimodality, and substantial areas with infeasible parametrizations (crash constraints) characterize control engineering tuning problems. Therefore, a flexible method to handle crash constraints with BO is presented. A resulting increase in sample efficiency is shown in comparison to standard BO. Furthermore, benchmark results indicate that pattern search (PS) performs best on a budget of 25 d objective function evaluations and a problem dimensionality d of d = 2. Bayesian adaptive direct search, a combination of BO and PS, is shown to be most sample efficient for 3 <= d <= 5. Using these optimizers instead of random search increases controller performance by on average 6.6% and up to 16.1%.Comment: 13 pages, 9 figure