Preference Exploration for Efficient Bayesian Optimization with Multiple Outcomes
This addresses the problem of efficiently optimizing experiments with multiple outcomes for decision-makers with unknown preferences, but it is incremental as it builds on existing Bayesian optimization and preference learning methods.
The paper tackles Bayesian optimization for expensive experiments with multiple outcomes where decision-maker preferences are unknown, by developing a framework that alternates between interactive preference learning and optimization, and demonstrates its performance through simulations.
We consider Bayesian optimization of expensive-to-evaluate experiments that generate vector-valued outcomes over which a decision-maker (DM) has preferences. These preferences are encoded by a utility function that is not known in closed form but can be estimated by asking the DM to express preferences over pairs of outcome vectors. To address this problem, we develop Bayesian optimization with preference exploration, a novel framework that alternates between interactive real-time preference learning with the DM via pairwise comparisons between outcomes, and Bayesian optimization with a learned compositional model of DM utility and outcomes. Within this framework, we propose preference exploration strategies specifically designed for this task, and demonstrate their performance via extensive simulation studies.