Sample-Efficient Differentially Private Fine-Tuning via Gradient Matrix Denoising
This work addresses the problem of slow optimization in private fine-tuning for language models, offering a domain-specific improvement that is incremental in nature.
The paper tackles the challenge of sample efficiency in differentially private fine-tuning of large language models by proposing a gradient matrix denoising algorithm based on random matrix theory, which improves sample efficiency and reduces training time on GLUE tasks with RoBERTa.
We address the challenge of sample efficiency in differentially private fine-tuning of large language models (LLMs) using DP-SGD. While DP-SGD provides strong privacy guarantees, the added noise significantly increases the entropy of gradient matrices, disrupting their low-rank structure and slowing optimization. We propose a post-processing algorithm that leverages random matrix theory to denoise gradients, restore low-rank structure, and improve alignment with the original signal. Applied to DP-SGD fine-tuning of RoBERTa on GLUE tasks, our method improves sample efficiency compared to state-of-the-art approaches, substantially reducing training time when optimal performance is not required. This work demonstrates that matrix recovery techniques can enhance the utility of private language model training without compromising privacy guarantees.