Evaluating Agents using Social Choice Theory
This provides a novel, interpretable evaluation method for AI agents and humans across domains like reinforcement learning and large language models, addressing cross-task evaluation challenges.
The paper tackles the problem of evaluating agents across tasks by framing it as a voting problem, using social choice theory to derive principled frameworks; it shows that the Voting-as-Evaluation approach is more robust than Elo and Nash averaging, predicts outcomes better in a seven-player game, and identifies properties like game-theoretic cycles.
We argue that many general evaluation problems can be viewed through the lens of voting theory. Each task is interpreted as a separate voter, which requires only ordinal rankings or pairwise comparisons of agents to produce an overall evaluation. By viewing the aggregator as a social welfare function, we are able to leverage centuries of research in social choice theory to derive principled evaluation frameworks with axiomatic foundations. These evaluations are interpretable and flexible, while avoiding many of the problems currently facing cross-task evaluation. We apply this Voting-as-Evaluation (VasE) framework across multiple settings, including reinforcement learning, large language models, and humans. In practice, we observe that VasE can be more robust than popular evaluation frameworks (Elo and Nash averaging), discovers properties in the evaluation data not evident from scores alone, and can predict outcomes better than Elo in a complex seven-player game. We identify one particular approach, maximal lotteries, that satisfies important consistency properties relevant to evaluation, is computationally efficient (polynomial in the size of the evaluation data), and identifies game-theoretic cycles.