VaultGemma: A Differentially Private Gemma Model
Amer Sinha, Thomas Mesnard, Ryan McKenna, Daogao Liu, Christopher A. Choquette-Choo, Yangsibo Huang, Da Yu, George Kaissis, Zachary Charles, Ruibo Liu, Lynn Chua, Pritish Kamath
arXiv:2510.15001v214 citationsh-index: 71
Originality Synthesis-oriented
AI Analysis
This addresses privacy concerns for users of large language models, though it appears incremental as it builds on the existing Gemma family.
The paper tackled the problem of training large language models with differential privacy by introducing VaultGemma 1B, a 1 billion parameter model fully trained with differential privacy, achieving a significant step forward in privacy-preserving capabilities.
We introduce VaultGemma 1B, a 1 billion parameter model within the Gemma family, fully trained with differential privacy. Pretrained on the identical data mixture used for the Gemma 2 series, VaultGemma 1B represents a significant step forward in privacy-preserving large language models. We openly release this model to the community