An Adaptive Approach to Bayesian Optimization with Setup Switching Costs

Black-box optimization methods typically assume that evaluations of the black-box objective function are equally costly to evaluate. We investigate here a resource-constrained setting where changes to certain decision variables of the search space incur a higher switching cost, e.g., due to expensive changes to the experimental setup. In this scenario, there is a trade-off between fixing the values of those costly variables or accepting this additional cost to explore more of the search space. We adapt two process-constrained batch algorithms to this sequential problem formulation, and propose two new methods — one cost-aware and one cost-ignorant. We validate and compare the algorithms using a set of 7 scalable test functions with different switching-cost settings. Our proposed cost-aware parameter-free algorithm yields comparable results to tuned process-constrained algorithms in all settings we considered, suggesting some degree of robustness to varying landscape features and cost trade-offs. This method starts to outperform the other algorithms with increasing switching cost. Our work expands on other recent Bayesian Optimization studies in resource-constrained settings that consider a batch setting only. Although the contributions of this work are relevant to the general class of resource-constrained problems, they are particularly relevant to problems where adaptability to varying resource availability is of high importance.