Just Sort It! A Simple and Effective Approach to Active Preference Learning
This provides an efficient solution for active preference learning, though it is incremental as it adapts existing sorting methods to a known problem.
The paper tackles the problem of learning a ranking from pairwise comparisons with minimal sampling by proposing a simple active learning strategy based on sorting algorithms like Quicksort, which achieves performance comparable to state-of-the-art methods at a much lower computational cost.
We address the problem of learning a ranking by using adaptively chosen pairwise comparisons. Our goal is to recover the ranking accurately but to sample the comparisons sparingly. If all comparison outcomes are consistent with the ranking, the optimal solution is to use an efficient sorting algorithm, such as Quicksort. But how do sorting algorithms behave if some comparison outcomes are inconsistent with the ranking? We give favorable guarantees for Quicksort for the popular Bradley-Terry model, under natural assumptions on the parameters. Furthermore, we empirically demonstrate that sorting algorithms lead to a very simple and effective active learning strategy: repeatedly sort the items. This strategy performs as well as state-of-the-art methods (and much better than random sampling) at a minuscule fraction of the computational cost.