Model Unlearning Objectives Vary for Distinct Language Functions
For LLM safety researchers, this work highlights the need for specialized unlearning approaches rather than one-size-fits-all methods, though it is an incremental step.
The paper argues that unlearning methods for LLMs should be tailored to specific language functions, demonstrating that distinct objectives for dangerous-knowledge and toxicity unlearning achieve strong results across four 7-8B models.
Large language models (LLMs) learn undesirable properties during pretraining, including dangerous knowledge and toxic text generation. Just as post-training uses different objectives to shape different behaviors, we argue that unlearning methods should be designed for the language function at issue. To study this, we consider two mechanistically distinct unlearning goals, dangerous-knowledge unlearning and toxicity unlearning. For dangerous knowledge, we introduce a cosine-based, meta-learned variant of RMU. For toxicity, we propose a multi-layer objective based on layer-specific probe directions. Across four open-source 7-8B models, our methods achieve strong results, based on distinct training objectives for the two types of unlearning. Overall, our results suggest that unlearning should be studied as a family of problems, analogous to the multiple types of LLM post-training.