Productionized Fairness Measurement Under Privacy Constraints
For tech companies needing to measure fairness across race/ethnicity while complying with privacy regulations, this provides a practical, privacy-preserving solution.
LinkedIn developed PPRE, a privacy-preserving method for estimating race/ethnicity to enable fairness measurements without exposing sensitive demographic data. The approach combines secure computation, differential privacy, and encryption to power disaggregated evaluations, demonstrating application on candidate- and viewer-side fairness metrics.
Fairness measurements in the form of disaggregated evaluations often rely on demographic signals that are legally constrained or culturally sensitive. Race and ethnicity signals are among the more difficult signals to curate and use for this task. This paper presents Privacy-Preserving Probabilistic Race/Ethnicity Estimation (PPRE) as a method for enabling fairness measurements with respect to race/ethnicity for U.S.\ LinkedIn members in a privacy-preserving manner. PPRE applies privacy technologies (specifically: secure two-party computation, differential privacy, and additive homomorphic encryption) on top of two race/ethnicity demographic signal sources (the Bayesian Improved Surname Geocoding estimator and a sparse golden survey set of self-reported demographics) to power a fairness measurement solution with respect to US-based race/ethnicity demographics. We detail its privacy guarantees and demonstrate its application on candidate- and viewer-side fairness measurements. We close with a transferable framework for institutions seeking to implement similar privacy-preserving measurement infrastructure.