Fair ranking: a critical review, challenges, and future directions
This work addresses the problem of fairness in ranking systems for stakeholders in online platforms and societal systems, but it is incremental as it builds on and critiques existing literature.
The paper critically reviews the existing fair ranking literature, highlighting its limitations in addressing context-specific concerns like the gap between high rankings and true utility, spillover effects, strategic incentives, and statistical uncertainty. It proposes a more holistic and impact-oriented research agenda to advance fair ranking.
Ranking, recommendation, and retrieval systems are widely used in online platforms and other societal systems, including e-commerce, media-streaming, admissions, gig platforms, and hiring. In the recent past, a large "fair ranking" research literature has been developed around making these systems fair to the individuals, providers, or content that are being ranked. Most of this literature defines fairness for a single instance of retrieval, or as a simple additive notion for multiple instances of retrievals over time. This work provides a critical overview of this literature, detailing the often context-specific concerns that such an approach misses: the gap between high ranking placements and true provider utility, spillovers and compounding effects over time, induced strategic incentives, and the effect of statistical uncertainty. We then provide a path forward for a more holistic and impact-oriented fair ranking research agenda, including methodological lessons from other fields and the role of the broader stakeholder community in overcoming data bottlenecks and designing effective regulatory environments.