On the Hidden Costs of Counterfactual Knowledge Training in LLM Unlearning
For researchers in LLM safety, this work identifies and diagnoses hidden costs of CFT, providing guidance for more rigorous unlearning research.
Counterfactual tuning (CFT) for LLM unlearning suffers from knowledge conflict and hallucination spillover, causing it to underperform other paradigms. The authors introduce RWKU+ benchmark to diagnose these issues.
Counterfactual tuning (CFT) has emerged as a promising paradigm for Large Language Model (LLM) unlearning by training models to generate alternative fictitious knowledge in place of undesired content. However, in this work, we find that this paradigm still underperforms other paradigms in some aspects, and identify two previously overlooked pitfalls underlying this gap: (1) knowledge conflict, where mutual inconsistencies within counterfactual corpora induce conflicting gradients that disrupt parameter optimization, and (2) hallucination spillover, where fitting false targets instills a persistent fabrication bias, inflating hallucination rates on unrelated domains. To systematically diagnose these issues, we introduce RWKU+, an extended benchmark equipped with novel trade-off metrics and gradient-level diagnostic tools. Our work further discusses the limitations and overhead of the paradigm, aiming to provide insights and actionable guidance for more rigorous LLM unlearning research.