Trade-Offs Between Fairness and Privacy in Language Modeling
This addresses the challenge of balancing fairness and privacy in NLP models, which is crucial for ethical AI deployment, but the work appears incremental as it builds on known trade-offs in classification tasks.
The paper investigates the trade-off between fairness and privacy in language models, finding that incorporating both privacy preservation and de-biasing techniques affects model utility and the relationship between these dimensions.
Protecting privacy in contemporary NLP models is gaining in importance. So does the need to mitigate social biases of such models. But can we have both at the same time? Existing research suggests that privacy preservation comes at the price of worsening biases in classification tasks. In this paper, we explore the extent to which this tradeoff really holds when we incorporate both privacy preservation and de-biasing techniques into training text generation models. How does improving the model along one dimension affect the other dimension as well as the utility of the model? We conduct an extensive set of experiments that include bias detection, privacy attacks, language modeling, and performance on downstream tasks.