Robust AI Evaluation through Maximal Lotteries
This addresses the issue of aggregating subjective preferences in AI evaluation for researchers and practitioners, offering a principled alternative to rankings for better serving diverse human preferences, though it is incremental as it builds on existing social choice theory.
The paper tackles the problem of evaluating language models on subjective tasks by showing that standard Bradley-Terry rankings force heterogeneous preferences into a total order, violating social-choice principles, and introduces robust lotteries that optimize worst-case performance, providing more reliable win rate guarantees across annotator distributions and recovering a stable set of top-performing models on large-scale preference datasets.
The standard way to evaluate language models on subjective tasks is through pairwise comparisons: an annotator chooses the "better" of two responses to a prompt. Leaderboards aggregate these comparisons into a single Bradley-Terry (BT) ranking, forcing heterogeneous preferences into a total order and violating basic social-choice desiderata. In contrast, social choice theory provides an alternative approach called maximal lotteries, which aggregates pairwise preferences without imposing any assumptions on their structure. However, we show that maximal lotteries are highly sensitive to preference heterogeneity and can favor models that severely underperform on specific tasks or user subpopulations. We introduce robust lotteries that optimize worst-case performance under plausible shifts in the preference data. On large-scale preference datasets, robust lotteries provide more reliable win rate guarantees across the annotator distribution and recover a stable set of top-performing models. By moving from rankings to pluralistic sets of winners, robust lotteries offer a principled step toward an ecosystem of complementary AI systems that serve the full spectrum of human preferences.