Current State in Privacy-Preserving Text Preprocessing for Domain-Agnostic NLP
This is an incremental review that highlights privacy risks in NLP for data practitioners and regulators.
The report addresses the challenge of protecting private information in textual data used for training large language models, by reviewing existing preprocessing approaches for anonymization in domain-gnostic NLP tasks.
Privacy is a fundamental human right. Data privacy is protected by different regulations, such as GDPR. However, modern large language models require a huge amount of data to learn linguistic variations, and the data often contains private information. Research has shown that it is possible to extract private information from such language models. Thus, anonymizing such private and sensitive information is of utmost importance. While complete anonymization may not be possible, a number of different pre-processing approaches exist for masking or pseudonymizing private information in textual data. This report focuses on a few of such approaches for domain-agnostic NLP tasks.