Fair NLP Models with Differentially Private Text Encoders
This addresses privacy and fairness concerns in NLP models for applications handling sensitive user data, representing an incremental advancement by integrating existing techniques.
The paper tackled the problem of sensitive attributes in text representations causing privacy and fairness issues in NLP models by proposing FEDERATE, which combines differential privacy and adversarial training to learn private and fairer text representations, showing consistent improvements over previous methods on four datasets.
Encoded text representations often capture sensitive attributes about individuals (e.g., race or gender), which raise privacy concerns and can make downstream models unfair to certain groups. In this work, we propose FEDERATE, an approach that combines ideas from differential privacy and adversarial training to learn private text representations which also induces fairer models. We empirically evaluate the trade-off between the privacy of the representations and the fairness and accuracy of the downstream model on four NLP datasets. Our results show that FEDERATE consistently improves upon previous methods, and thus suggest that privacy and fairness can positively reinforce each other.