Fine-Tuning Large Language Models with User-Level Differential Privacy
This work addresses privacy concerns in fine-tuning LLMs for users by providing scalable algorithms, though it is incremental as it builds on existing DP-SGD methods.
The paper tackled the problem of training large language models with user-level differential privacy to protect all examples from each user, showing that user-level sampling generally yields better results than example-level sampling, especially under strong privacy or large compute budgets, with experiments scaling to models with hundreds of millions of parameters.
We investigate practical and scalable algorithms for training large language models (LLMs) with user-level differential privacy (DP) in order to provably safeguard all the examples contributed by each user. We study two variants of DP-SGD with: (1) example-level sampling (ELS) and per-example gradient clipping, and (2) user-level sampling (ULS) and per-user gradient clipping. We derive a novel user-level DP accountant that allows us to compute provably tight privacy guarantees for ELS. Using this, we show that while ELS can outperform ULS in specific settings, ULS generally yields better results when each user has a diverse collection of examples. We validate our findings through experiments in synthetic mean estimation and LLM fine-tuning tasks under fixed compute budgets. We find that ULS is significantly better in settings where either (1) strong privacy guarantees are required, or (2) the compute budget is large. Notably, our focus on LLM-compatible training algorithms allows us to scale to models with hundreds of millions of parameters and datasets with hundreds of thousands of users.