Parallel Sequential Random Embedding Bayesian Optimization

Abstract

Bayesian optimization, which offers efficient parameter search, suffers from high computation cost if the parameters have high dimensionality because the search space expands and more trials are needed. One existing solution is an embedding method that enables the search to be restricted to a low-dimensional subspace, but this method works well only when the number of embedding dimensions closely matches the number of effective dimensions, which affects the function value. However, in practical situations, the number of effective dimensions is unknown, and using a low dimensional subspace to lower computation costs often results in less effective searches. This study proposes a Bayesian optimization method that uses random embedding that remains efficient even if the embedded dimension is lower than the effective dimensions. By conducting parallel search in an initially low dimensional space and performing multiple cycles in which the search space is incrementally improved, the optimum solution can be efficiently found. The proposed method is challenged in experiments on benchmark problems, the results of which confirm its effectiveness.