Patching LLM Like Software: A Lightweight Method for Improving Safety Policy in Large Language Models
This provides a practical mechanism for vendors and practitioners to efficiently address safety gaps in LLMs between major releases, though it is incremental as it builds on existing fine-tuning and prefix-tuning approaches.
The paper tackles the problem of safety vulnerabilities in large language models (LLMs) by proposing a lightweight patching method that prepends a compact prefix to existing models, achieving safety improvements comparable to next-generation models with only 0.003% additional parameters.
We propose patching for large language models (LLMs) like software versions, a lightweight and modular approach for addressing safety vulnerabilities. While vendors release improved LLM versions, major releases are costly, infrequent, and difficult to tailor to customer needs, leaving released models with known safety gaps. Unlike full-model fine-tuning or major version updates, our method enables rapid remediation by prepending a compact, learnable prefix to an existing model. This "patch" introduces only 0.003% additional parameters, yet reliably steers model behavior toward that of a safer reference model. Across three critical domains (toxicity mitigation, bias reduction, and harmfulness refusal) policy patches achieve safety improvements comparable to next-generation safety-aligned models while preserving fluency. Our results demonstrate that LLMs can be "patched" much like software, offering vendors and practitioners a practical mechanism for distributing scalable, efficient, and composable safety updates between major model releases.