SDD: Self-Degraded Defense against Malicious Fine-tuning
This addresses a security vulnerability in open-source LLMs for developers and users, but it is incremental as it builds on existing safety alignment methods.
The paper tackles the problem of malicious fine-tuning bypassing safety alignment in open-source LLMs by introducing the Self-Degraded Defense (SDD) framework, which causes LLMs to produce irrelevant responses to harmful prompts and significantly degrades their general capability during malicious fine-tuning, preventing harmful instruction following.
Open-source Large Language Models (LLMs) often employ safety alignment methods to resist harmful instructions. However, recent research shows that maliciously fine-tuning these LLMs on harmful data can easily bypass these safeguards. To counter this, we theoretically uncover why malicious fine-tuning succeeds and identify potential defense strategies. Building on the theoretical analysis, we introduce the Self-Degraded Defense (SDD) framework. SDD encourages LLMs to produce high-quality but irrelevant responses to harmful prompts. When attackers attempt malicious fine-tuning, the general capability of the LLM aligned by SDD will significantly decrease, rendering it incapable of following harmful instructions. Our experimental results confirm SDD's effectiveness against such attacks.