On the Vulnerability of Parameter-Level Defenses to Model Merging
For the security community, this work reveals a fundamental weakness in current defenses against unauthorized model merging, but the proposed countermeasure suggests an incremental advance rather than a breakthrough.
The paper identifies a vulnerability in parameter-level defenses against model merging, showing that protected task vectors are dominated by the pretrained model. The authors propose Anchor-Guided Attack (AGA) to bypass these defenses and Anchor-Repulsive Fine-tuning (ARF) to mitigate the attack.
The training-free integration of expert models via model merging has exposed significant security risks, enabling free-riders to combine specialized models without authorization. Recent works propose parameter-level defenses that employ linear parameter transformations to neutralize this threat. In this paper, we systematically analyze such defenses and reveal that their protected task vectors are inherently small in magnitude. Consequently, the protected weights remain overwhelmingly dominated by the pretrained model. Based on this observation, we designate the pretrained model as a static reference anchor and propose the Anchor-Guided Attack (AGA) to circumvent existing safeguards. Specifically, AGA aligns the protected model with this anchor to recover the transformation matrix analytically. Extensive evaluations validate that AGA consistently bypasses both individual and composite defenses under realistic defense-agnostic scenarios. Furthermore, we provide Anchor-Repulsive Fine-tuning (ARF), a defense method to mitigate the anchor dominance leveraged by AGA. Empirical results confirm that ARF effectively defeats the proposed attack. Our code is available at https://github.com/krumpguo/secure-merge-attack.