Look-Ahead Acquisition Functions for Bernoulli Level Set Estimation
This addresses efficient active sampling for binary-response applications like psychophysics, representing an incremental advance by adapting continuous-output methods to Bernoulli outcomes.
The paper tackled the problem of level set estimation with binary outcomes, common in human psychophysics, by deriving analytic expressions for look-ahead acquisition functions using Gaussian process classification, and demonstrated benefits on benchmark and real-world tasks.
Level set estimation (LSE) is the problem of identifying regions where an unknown function takes values above or below a specified threshold. Active sampling strategies for efficient LSE have primarily been studied in continuous-valued functions. Motivated by applications in human psychophysics where common experimental designs produce binary responses, we study LSE active sampling with Bernoulli outcomes. With Gaussian process classification surrogate models, the look-ahead model posteriors used by state-of-the-art continuous-output methods are intractable. However, we derive analytic expressions for look-ahead posteriors of sublevel set membership, and show how these lead to analytic expressions for a class of look-ahead LSE acquisition functions, including information-based methods. Benchmark experiments show the importance of considering the global look-ahead impact on the entire posterior. We demonstrate a clear benefit to using this new class of acquisition functions on benchmark problems, and on a challenging real-world task of estimating a high-dimensional contrast sensitivity function.