Efficient Model Editing with Task-Localized Sparse Fine-tuning
This work addresses the problem of efficient and conflict-free model editing for adaptable foundation models, representing an incremental improvement over existing task arithmetic methods.
The paper tackles the computational bottlenecks and lack of weight disentanglement in task arithmetic for model editing by proposing TaLoS, which uses sparse fine-tuning on low-sensitivity parameters, resulting in improved training and inference efficiency and outperforming current methods in task addition and negation.
Task arithmetic has emerged as a promising approach for editing models by representing task-specific knowledge as composable task vectors. However, existing methods rely on network linearization to derive task vectors, leading to computational bottlenecks during training and inference. Moreover, linearization alone does not ensure weight disentanglement, the key property that enables conflict-free composition of task vectors. To address this, we propose TaLoS which allows to build sparse task vectors with minimal interference without requiring explicit linearization and sharing information across tasks. We find that pre-trained models contain a subset of parameters with consistently low gradient sensitivity across tasks, and that sparsely updating only these parameters allows for promoting weight disentanglement during fine-tuning. Our experiments prove that TaLoS improves training and inference efficiency while outperforming current methods in task addition and negation. By enabling modular parameter editing, our approach fosters practical deployment of adaptable foundation models in real-world applications.