BetaEdit: Null-Space Constrained Sequential Model Editing
For practitioners of model editing, this work addresses the critical bottleneck of performance decline in long-horizon sequential edits, offering a refined null-space method that preserves both editing accuracy and general capabilities.
The paper identifies knowledge leakage in null-space-based model editing methods and proposes BetaEdit, which integrates history-aware updates to mitigate performance degradation during sequential editing. BetaEdit consistently outperforms prior methods on massive-scale sequential editing across three LLMs and two benchmarks.
Null-space-based methods have garnered considerable attention in model editing by constraining updates to the null space of the pre-existing knowledge representation, thereby preserving the model's original behavior. However, in practice these methods rely on an approximate null space--leading to knowledge leakage--and further suffer from severe performance degradation during sequential editing. Recent work shows that history-aware editing strategies can empirically mitigate this decline, yet the underlying reason remains unclear. In this paper, we first expose the knowledge leakage inherent in existing null-space approaches and then analyze why history-aware updates effectively preserve both editing performance and general capabilities during long-horizon editing. Building on these insights, we propose BetaEdit, a refined framework that effectively controls the knowledge leakage and integrates history-aware updates into the null-space paradigm. Extensive experiments on three large language models across two standard benchmarks show that BetaEdit consistently outperforms prior methods in the challenging regime of massive-scale sequential editing. Code is available at: https://github.com/lbq8942/BetaEdit.