Selective Ensemble Based on Preference-Directed Multi-Objective Bandits
For practitioners needing to select models under limited evaluation budgets and partial preference specifications, this provides a principled framework with theoretical guarantees.
The paper formalizes selective ensemble under partial preferences as a preference-directed multi-objective bandit problem, introduces the PrefUCB algorithm, and proves instance-dependent logarithmic regret bounds. Experiments on model ensemble and asset allocation tasks validate the method.
Selective ensemble for modern machine learning systems requires choosing promising model candidates under limited evaluation budgets, while downstream tasks often specify only partial preferences over capabilities such as accuracy, robustness, and reasoning. This setting naturally gives rise to a sequential decision problem under partially specified linear preferences. We formalize it as preference-directed multi-objective bandits (PDMOB), where admissible trade-offs are represented by a polyhedral preference cone. Based on this formulation, we introduce Pareto $C$-optimality, which recovers standard Pareto optimality and single-weight scalarization as special cases. We then propose the preference-directed upper confidence bound (PrefUCB) algorithm, which maintains directional confidence intervals to guide exploration. We analyze both indicator-based and gap-weighted regret, and establish instance-dependent logarithmic bounds for both criteria, recovering the optimal logarithmic dependence on the horizon $T$ in classical special cases. Experiments on large pre-trained model selective ensemble tasks and online asset allocation under institutional mandates validate the efficacy of our method.