Testing Group Fairness via Optimal Transport Projections
This work addresses the need for statistically rigorous auditing of fairness in ML algorithms, which is crucial for ensuring ethical AI deployment in domains like hiring or lending, though it is incremental in building on existing fairness testing methods.
The authors tackled the problem of detecting violations of group fairness in machine learning classifiers by developing a statistical testing framework based on optimal transport projections. The result is a flexible, interpretable tool that efficiently computes test statistics using linear programming and provides asymptotic distributions for rigorous auditing.
We present a statistical testing framework to detect if a given machine learning classifier fails to satisfy a wide range of group fairness notions. The proposed test is a flexible, interpretable, and statistically rigorous tool for auditing whether exhibited biases are intrinsic to the algorithm or due to the randomness in the data. The statistical challenges, which may arise from multiple impact criteria that define group fairness and which are discontinuous on model parameters, are conveniently tackled by projecting the empirical measure onto the set of group-fair probability models using optimal transport. This statistic is efficiently computed using linear programming and its asymptotic distribution is explicitly obtained. The proposed framework can also be used to test for testing composite fairness hypotheses and fairness with multiple sensitive attributes. The optimal transport testing formulation improves interpretability by characterizing the minimal covariate perturbations that eliminate the bias observed in the audit.