Private Fine-tuning of Large Language Models with Zeroth-order Optimization
This work addresses privacy-preserving fine-tuning for large language models, offering a more memory-efficient alternative to DP-SGD, though it is incremental as it builds on existing zeroth-order optimization methods.
The paper tackles the challenge of scaling differentially private training to large language models by introducing DP-ZO, a private fine-tuning framework based on zeroth-order optimization that privatizes only a scalar step size, achieving a strong privacy-utility trade-off comparable to DP-SGD with improved memory efficiency and higher utility under Laplace mechanism.
Differentially private stochastic gradient descent (DP-SGD) allows models to be trained in a privacy-preserving manner, but has proven difficult to scale to the era of foundation models. We introduce DP-ZO, a private fine-tuning framework for large language models by privatizing zeroth order optimization methods. A key insight into the design of our method is that the direction of the gradient in the zeroth-order optimization we use is random and the only information from training data is the step size, i.e., a scalar. Therefore, we only need to privatize the scalar step size, which is memory-efficient. DP-ZO provides a strong privacy-utility trade-off across different tasks, and model sizes that are comparable to DP-SGD in $(\varepsilon,δ)$-DP. Notably, DP-ZO possesses significant advantages over DP-SGD in memory efficiency, and obtains higher utility in $\varepsilon$-DP when using the Laplace mechanism.