BINOCULARS for Efficient, Nonmyopic Sequential Experimental Design
This addresses the problem of underweighting exploration in SED for researchers and practitioners in fields like hyperparameter tuning, though it is an incremental improvement over existing approximations.
The paper tackles the intractability of optimal sequential experimental design (SED) by proposing BINOCULARS, a framework for efficient nonmyopic approximations, and demonstrates that it significantly outperforms myopic alternatives in real-world scenarios like Bayesian optimization and quadrature.
Finite-horizon sequential experimental design (SED) arises naturally in many contexts, including hyperparameter tuning in machine learning among more traditional settings. Computing the optimal policy for such problems requires solving Bellman equations, which are generally intractable. Most existing work resorts to severely myopic approximations by limiting the decision horizon to only a single time-step, which can underweight exploration in favor of exploitation. We present BINOCULARS: Batch-Informed NOnmyopic Choices, Using Long-horizons for Adaptive, Rapid SED, a general framework for deriving efficient, nonmyopic approximations to the optimal experimental policy. Our key idea is simple and surprisingly effective: we first compute a one-step optimal batch of experiments, then select a single point from this batch to evaluate. We realize BINOCULARS for Bayesian optimization and Bayesian quadrature -- two notable SED problems with radically different objectives -- and demonstrate that BINOCULARS significantly outperforms myopic alternatives in real-world scenarios.