Towards Fair RAG: On the Impact of Fair Ranking in Retrieval-Augmented Generation
This addresses fairness for stakeholders in RAG systems, though it is incremental as it applies existing fair ranking concepts to RAG.
The paper tackles the problem of fairness in retrieval-augmented generation (RAG) systems by evaluating the impact of fair ranking on both retrieval and generation quality, showing that incorporating fairness-aware retrieval often maintains or enhances performance across twelve models and seven tasks while ensuring more balanced source attribution.
Despite the central role of retrieval in retrieval-augmented generation (RAG) systems, much of the existing research on RAG overlooks the well-established field of fair ranking and fails to account for the interests of all stakeholders involved. In this paper, we conduct the first systematic evaluation of RAG systems that integrate fairness-aware rankings, addressing both ranking fairness and attribution fairness, which ensures equitable exposure of the sources cited in the generated content. Our evaluation focuses on measuring item-side fairness, specifically the fair exposure of relevant items retrieved by RAG systems, and investigates how this fairness impacts both the effectiveness of the systems and the attribution of sources in the generated output that users ultimately see. By experimenting with twelve RAG models across seven distinct tasks, we show that incorporating fairness-aware retrieval often maintains or even enhances both ranking quality and generation quality, countering the common belief that fairness compromises system performance. Additionally, we demonstrate that fair retrieval practices lead to more balanced attribution in the final responses, ensuring that the generator fairly cites the sources it relies on. Our findings underscore the importance of item-side fairness in retrieval and generation, laying the foundation for responsible and equitable RAG systems and guiding future research in fair ranking and attribution.