Questioning the assumptions behind fairness solutions
This work addresses the limitations of current fairness frameworks for populations impacted by biased optimization systems, highlighting an incremental shift towards user-centric protections.
The paper critiques the assumptions underlying fairness solutions in optimization systems, arguing that service providers often lack the incentives or means to mitigate negative externalities, and proposes Protective Optimization Technologies as a defense for affected subjects.
In addition to their benefits, optimization systems can have negative economic, moral, social, and political effects on populations as well as their environments. Frameworks like fairness have been proposed to aid service providers in addressing subsequent bias and discrimination during data collection and algorithm design. However, recent reports of neglect, unresponsiveness, and malevolence cast doubt on whether service providers can effectively implement fairness solutions. These reports invite us to revisit assumptions made about the service providers in fairness solutions. Namely, that service providers have (i) the incentives or (ii) the means to mitigate optimization externalities. Moreover, the environmental impact of these systems suggests that we need (iii) novel frameworks that consider systems other than algorithmic decision-making and recommender systems, and (iv) solutions that go beyond removing related algorithmic biases. Going forward, we propose Protective Optimization Technologies that enable optimization subjects to defend against negative consequences of optimization systems.