Consecutive Preferential Bayesian Optimization
This work addresses the challenge of optimizing expensive-to-produce and evaluate objectives for human experts in preference-based optimization, representing an incremental improvement by generalizing existing methods to account for costs and ambiguity.
The paper tackles the problem of high production and evaluation costs in preferential Bayesian optimization by introducing Consecutive Preferential Bayesian Optimization, which constrains comparisons to previously generated candidates and incorporates a Just-Noticeable Difference threshold to handle perceptual ambiguity, resulting in a notable increase in accuracy in setups with high costs or indifference feedback.
Preferential Bayesian optimization allows optimization of objectives that are either expensive or difficult to measure directly, by relying on a minimal number of comparative evaluations done by a human expert. Generating candidate solutions for evaluation is also often expensive, but this cost is ignored by existing methods. We generalize preference-based optimization to explicitly account for production and evaluation costs with Consecutive Preferential Bayesian Optimization, reducing production cost by constraining comparisons to involve previously generated candidates. We also account for the perceptual ambiguity of the oracle providing the feedback by incorporating a Just-Noticeable Difference threshold into a probabilistic preference model to capture indifference to small utility differences. We adapt an information-theoretic acquisition strategy to this setting, selecting new configurations that are most informative about the unknown optimum under a preference model accounting for the perceptual ambiguity. We empirically demonstrate a notable increase in accuracy in setups with high production costs or with indifference feedback.