Fairness in machine learning: against false positive rate equality as a measure of fairness
This work critiques a foundational fairness measure in ML, potentially reshaping how bias is evaluated in consequential decisions, though it is incremental in ethics-focused analysis.
The paper argues that false positive rate equality, a popular fairness metric in machine learning, does not actually measure fairness, challenging its use in algorithmic bias assessments. It provides an ethical framework to show this metric sets an incoherent standard, addressing a key assumption in fairness tradeoffs.
As machine learning informs increasingly consequential decisions, different metrics have been proposed for measuring algorithmic bias or unfairness. Two popular fairness measures are calibration and equality of false positive rate. Each measure seems intuitively important, but notably, it is usually impossible to satisfy both measures. For this reason, a large literature in machine learning speaks of a fairness tradeoff between these two measures. This framing assumes that both measures are, in fact, capturing something important. To date, philosophers have not examined this crucial assumption, and examined to what extent each measure actually tracks a normatively important property. This makes this inevitable statistical conflict, between calibration and false positive rate equality, an important topic for ethics. In this paper, I give an ethical framework for thinking about these measures and argue that, contrary to initial appearances, false positive rate equality does not track anything about fairness, and thus sets an incoherent standard for evaluating the fairness of algorithms.