Addressing multiple metrics of group fairness in data-driven decision making
This addresses fairness in data-driven decisions for socio-demographic groups, but it is incremental as it builds on existing metrics without new algorithmic solutions.
The paper tackles the problem of multiple group fairness metrics in machine learning being incompatible and proposes using Principal Component Analysis (PCA) to visualize and cluster these metrics, showing that PCA explains variance with one to three components across datasets.
The Fairness, Accountability, and Transparency in Machine Learning (FAT-ML) literature proposes a varied set of group fairness metrics to measure discrimination against socio-demographic groups that are characterized by a protected feature, such as gender or race.Such a system can be deemed as either fair or unfair depending on the choice of the metric. Several metrics have been proposed, some of them incompatible with each other.We do so empirically, by observing that several of these metrics cluster together in two or three main clusters for the same groups and machine learning methods. In addition, we propose a robust way to visualize multidimensional fairness in two dimensions through a Principal Component Analysis (PCA) of the group fairness metrics. Experimental results on multiple datasets show that the PCA decomposition explains the variance between the metrics with one to three components.