Retrievable Gradients: Continual Post-Training Without Cumulative Weight Drift
This work addresses the problem of catastrophic forgetting and weight drift in continual post-training for large language models, offering a scalable and reversible method for parametric knowledge injection.
ReGrad introduces a new paradigm for continual post-training that treats gradients as retrievable units, storing document-specific gradients in a Gradient Bank and retrieving query-relevant ones at inference time for temporary weight adaptation, outperforming CPT and RAG baselines without cumulative weight drift.
Continual post-training enables models to absorb emerging knowledge after deployment, but repeatedly updating shared parameters can accumulate weight drift, potentially causing catastrophic forgetting and degrading general capabilities. Retrieval-augmented generation avoids such parameter drift, yet often lacks the depth of parametric knowledge integration. In this paper, we propose ReGrad (Retrievable Gradients), a new paradigm that treats gradients as retrievable units of knowledge. ReGrad pre-computes document-specific gradients offline, stores them in an indexed Gradient Bank, and retrieves only query-relevant gradients at inference time for temporary weight adaptation. However, raw language-modeling gradients are optimized for token-level document reconstruction rather than for query-driven knowledge use. We therefore introduce a bi-level meta-learning objective that reshapes document-derived gradients into generalizable adaptation signals for downstream tasks. Experiments across general and domain-specific settings show that \textsc{ReGrad} outperforms CPT and RAG baselines, enabling scalable and reversible parametric knowledge injection without accumulating weight drift.