Exposing the Illusion of Fairness: Auditing Vulnerabilities to Distributional Manipulation Attacks
This exposes a critical vulnerability in regulation-driven fairness audits for AI systems, which is an incremental but important contribution to ensuring compliance with laws like the EU AI Act.
The paper tackles the problem of AI fairness audits being vulnerable to manipulation by showing how data samples can be minimally perturbed to artificially satisfy fairness criteria like Disparate Impact, while remaining statistically indistinguishable from the original distribution, and provides methods to detect such manipulations.
Proving the compliance of AI algorithms has become an important challenge with the growing deployment of such algorithms for real-life applications. Inspecting possible biased behaviors is mandatory to satisfy the constraints of the regulations of the EU Artificial Intelligence's Act. Regulation-driven audits increasingly rely on global fairness metrics, with Disparate Impact being the most widely used. Yet such global measures depend highly on the distribution of the sample on which the measures are computed. We investigate first how to manipulate data samples to artificially satisfy fairness criteria, creating minimally perturbed datasets that remain statistically indistinguishable from the original distribution while satisfying prescribed fairness constraints. Then we study how to detect such manipulation. Our analysis (i) introduces mathematically sound methods for modifying empirical distributions under fairness constraints using entropic or optimal transport projections, (ii) examines how an auditee could potentially circumvent fairness inspections, and (iii) offers recommendations to help auditors detect such data manipulations. These results are validated through experiments on classical tabular datasets in bias detection.