Amifa Raj

Amifa Raj, Ashlee Milton, Michael D. Ekstrand

In this position paper, we argue for the need to investigate if and how gender stereotypes manifest in search and recommender systems.As a starting point, we particularly focus on how these systems may propagate and reinforce gender stereotypes through their results in learning environments, a context where teachers and children in their formative stage regularly interact with these systems. We provide motivating examples supporting our concerns and outline an agenda to support future research addressing the phenomena.

Amifa Raj, Michael D. Ekstrand

Ranked lists are frequently used by information retrieval (IR) systems to present results believed to be relevant to the users information need. Fairness is a relatively new but important aspect of these rankings to measure, joining a rich set of metrics that go beyond traditional accuracy or utility constructs to provide a more holistic understanding of IR system behavior. In the last few years, several metrics have been proposed to quantify the (un)fairness of rankings, particularly with respect to particular group(s) of content providers, but comparative analyses of these metrics -- particularly for IR -- is lacking. There is limited guidance, therefore, to decide what fairness metrics are applicable to a specific scenario, or assessment of the extent to which metrics agree or disagree applied to real data. In this paper, we describe several fair ranking metrics from existing literature in a common notation, enabling direct comparison of their assumptions, goals, and design choices; we then empirically compare them on multiple data sets covering both search and recommendation tasks.