Fairness Issues and Mitigations in (Differentially Private) Socio-Demographic Data Processes
This addresses fairness problems in policy-making and resource allocation for statistical agencies, with incremental contributions to mitigating biases in sampling and privacy methods.
The paper tackles fairness issues in socio-demographic data sampling, showing that sampling errors unevenly impact group-level estimates and compromise fairness in decisions, and finds that differential privacy noise can reduce unfairness by positively biasing smaller counts, validated with census datasets.
Statistical agencies rely on sampling techniques to collect socio-demographic data crucial for policy-making and resource allocation. This paper shows that surveys of important societal relevance introduce sampling errors that unevenly impact group-level estimates, thereby compromising fairness in downstream decisions. To address these issues, this paper introduces an optimization approach modeled on real-world survey design processes, ensuring sampling costs are optimized while maintaining error margins within prescribed tolerances. Additionally, privacy-preserving methods used to determine sampling rates can further impact these fairness issues. This paper explores the impact of differential privacy on the statistics informing the sampling process, revealing a surprising effect: not only is the expected negative effect from the addition of noise for differential privacy negligible, but also this privacy noise can in fact reduce unfairness as it positively biases smaller counts. These findings are validated over an extensive analysis using datasets commonly applied in census statistics.