A Note on the Significance Adjustment for FA*IR with Two Protected Groups
This work provides an incremental improvement to the FA*IR algorithm, benefiting researchers and practitioners working on fair ranking systems, particularly in scenarios with short rankings or low representation of protected groups.
The authors improved the significance adjustment for the FA*IR algorithm, which previously failed for short rankings and low minimum protected group proportions. They developed a method to exactly calculate the minimum number of protected candidates per ranking position and introduced a mapping from continuous significance levels to discrete tables, enabling binary search for the critical significance level.
In this report we provide an improvement of the significance adjustment from the FA*IR algorithm of Zehlike et al., which did not work for very short rankings in combination with a low minimum proportion $p$ for the protected group. We show how the minimum number of protected candidates per ranking position can be calculated exactly and provide a mapping from the continuous space of significance levels ($α$) to a discrete space of tables, which allows us to find $α_c$ using a binary search heuristic.