Fair Inputs and Fair Outputs: The Incompatibility of Fairness in Privacy and Accuracy
This addresses fairness concerns in algorithmic decision-making for stakeholders like policymakers and developers, but it is incremental as it builds on existing fairness frameworks.
The paper tackles the incompatibility between fairness in inputs (fair privacy and need-to-know) and fairness in outputs (fair prediction accuracy) for optimal classifiers, showing that these properties are generally incompatible and common in real data.
Fairness concerns about algorithmic decision-making systems have been mainly focused on the outputs (e.g., the accuracy of a classifier across individuals or groups). However, one may additionally be concerned with fairness in the inputs. In this paper, we propose and formulate two properties regarding the inputs of (features used by) a classifier. In particular, we claim that fair privacy (whether individuals are all asked to reveal the same information) and need-to-know (whether users are only asked for the minimal information required for the task at hand) are desirable properties of a decision system. We explore the interaction between these properties and fairness in the outputs (fair prediction accuracy). We show that for an optimal classifier these three properties are in general incompatible, and we explain what common properties of data make them incompatible. Finally we provide an algorithm to verify if the trade-off between the three properties exists in a given dataset, and use the algorithm to show that this trade-off is common in real data.