Tuning without Peeking: Provable Privacy and Generalization Bounds for LLM Post-Training
This work addresses privacy and security concerns for LLM deployment in sensitive environments, offering a novel method with strong theoretical guarantees, though it is incremental as an add-on to existing gradient-based approaches.
The paper tackles the problem of privacy and security risks in gradient-based LLM post-training by introducing BBoxER, an evolutionary black-box optimization method that provides theoretical guarantees for differential privacy and robustness to attacks, demonstrating improved performance and generalization on reasoning benchmarks with a few iterations.
Gradient-based optimization is the workhorse of deep learning, offering efficient and scalable training via backpropagation. However, exposing gradients during training can leak sensitive information about the underlying data, raising privacy and security concerns such as susceptibility to data poisoning attacks. In contrast, black box optimization methods, which treat the model as an opaque function, relying solely on function evaluations to guide optimization, offer a promising alternative in scenarios where data access is restricted, adversarial risks are high, or overfitting is a concern. This paper introduces BBoxER, an evolutionary black-box method for LLM post-training that induces an information bottleneck via implicit compression of the training data. Leveraging the tractability of information flow, we provide non-vacuous generalization bounds and strong theoretical guarantees for differential privacy, robustness to data poisoning attacks, and extraction attacks. In experiments with LLMs, we demonstrate empirically that black-box optimization methods-despite the scalability and computational challenges inherent to black-box approaches-are able to learn, showing how a few iterations of BBoxER improve performance, generalize well on a benchmark of reasoning datasets, and are robust to membership inference attacks. This positions BBoxER as an attractive add-on on top of gradient-based optimization, offering suitability for deployment in restricted or privacy-sensitive environments while also providing non-vacuous generalization guarantees.