Behind the Mask: Demographic bias in name detection for PII masking
This work highlights demographic bias in privacy tools, posing risks of unequal privacy protection for marginalized groups, and is incremental in benchmarking existing systems.
The paper evaluated three off-the-shelf PII masking systems for name detection and found that all systems showed significant demographic biases, with the highest error rates for names associated with Black and Asian/Pacific Islander individuals, though an open-source RoBERTa-based system had fewer disparities.
Many datasets contain personally identifiable information, or PII, which poses privacy risks to individuals. PII masking is commonly used to redact personal information such as names, addresses, and phone numbers from text data. Most modern PII masking pipelines involve machine learning algorithms. However, these systems may vary in performance, such that individuals from particular demographic groups bear a higher risk for having their personal information exposed. In this paper, we evaluate the performance of three off-the-shelf PII masking systems on name detection and redaction. We generate data using names and templates from the customer service domain. We find that an open-source RoBERTa-based system shows fewer disparities than the commercial models we test. However, all systems demonstrate significant differences in error rate based on demographics. In particular, the highest error rates occurred for names associated with Black and Asian/Pacific Islander individuals.