Resolving Undesired Bias in Optimization of Environmentally Adaptive Control Policies

Significant research on experiment-based black-box optimization using Bayesian optimization techniques is being performed because of its usefulness in a wide range of fields. Several algorithms using Bayesian optimization for optimizing environmentally adaptive control policies have been developed. This adaptivity is expected to be crucial for applications such as mobile robots. In this work, the unbiased expected improvement metric was the key to efficiently obtain the approximated optimal policy. The purpose of the metric was to remove the bias in sample points that is inevitable if ordinary metrics, such as the expected improvement, are used. This paper clarified the mechanism that causes the bias and showed that the bias should be attenuated to achieve efficient experiments. Based on the understanding of the mechanism, a simple solution was proposed to attenuate this bias. Using numerical tests, it was shown that our method effectively attenuated the bias and that this led to better optimization performance in that it often required less samples than the existing method