From Fragile to Certified: Wasserstein Audits of Group Fairness Under Distribution Shift
This provides a principled method for auditing and certifying group fairness in machine learning models under distribution shift, addressing a critical reliability issue for practitioners and regulators.
The paper tackles the problem of unreliable group-fairness audits under distribution shift by proposing a Wasserstein distributionally robust framework that certifies worst-case fairness, delivering stable assessments across benchmarks and classifiers.
Group-fairness metrics (e.g., equalized odds) can vary sharply across resamples and are especially brittle under distribution shift, undermining reliable audits. We propose a Wasserstein distributionally robust framework that certifies worst-case group fairness over a ball of plausible test distributions centered at the empirical law. Our formulation unifies common group fairness notions via a generic conditional-probability functional and defines $\varepsilon$-Wasserstein Distributional Fairness ($\varepsilon$-WDF) as the audit target. Leveraging strong duality, we derive tractable reformulations and an efficient estimator (DRUNE) for $\varepsilon$-WDF. We prove feasibility and consistency and establish finite-sample certification guarantees for auditing fairness, along with quantitative bounds under smoothness and margin conditions. Across standard benchmarks and classifiers, $\varepsilon$-WDF delivers stable fairness assessments under distribution shift, providing a principled basis for auditing and certifying group fairness beyond observational data.