You Are What You Write: Preserving Privacy in the Era of Large Language Models
This addresses privacy risks for users of large language models, but it is incremental as it builds on existing privacy-preserving methods.
The paper investigates the extent of personal information leakage in pre-trained large language models, finding a positive correlation between model complexity, pre-training data size, and data leakage, and evaluates privacy-preserving algorithms on a multilingual sentiment analysis dataset, showing that differential privacy can reduce utility but hybrid techniques help mitigate this.
Large scale adoption of large language models has introduced a new era of convenient knowledge transfer for a slew of natural language processing tasks. However, these models also run the risk of undermining user trust by exposing unwanted information about the data subjects, which may be extracted by a malicious party, e.g. through adversarial attacks. We present an empirical investigation into the extent of the personal information encoded into pre-trained representations by a range of popular models, and we show a positive correlation between the complexity of a model, the amount of data used in pre-training, and data leakage. In this paper, we present the first wide coverage evaluation and comparison of some of the most popular privacy-preserving algorithms, on a large, multi-lingual dataset on sentiment analysis annotated with demographic information (location, age and gender). The results show since larger and more complex models are more prone to leaking private information, use of privacy-preserving methods is highly desirable. We also find that highly privacy-preserving technologies like differential privacy (DP) can have serious model utility effects, which can be ameliorated using hybrid or metric-DP techniques.