CE-U: Cross Entropy Unlearning
This addresses privacy concerns in AI by enabling effective unlearning of sensitive data, though it is incremental as it builds on existing unlearning frameworks.
The authors tackled the problem of large language models memorizing sensitive data by proposing CE-U, a cross entropy unlearning loss function that addresses gradient issues in existing methods and achieved state-of-the-art results on the TOFU benchmark for LLaMA2-7B models without extra resources.
Large language models memorize sensitive data from their pretraining corpora. In this work, we propose CE-U (Cross Entropy Unlearning), a loss function for unlearning. CE-U addresses fundamental limitations of gradient ascent approaches that suffer from vanishing gradients when model confidence is high and exploding gradients when confidence is low. We also unify standard cross entropy learning and unlearning into a single framework. On the TOFU benchmark for unlearning, CE-U achieves state-of-the-art results on LLaMA2-7B models without using an extra oracle model or additional positive samples. Our analysis reveals that the problematic gradient ascent component also exists in reinforcement learning algorithms like DPO and GRPO. This suggests that applying CE-U approach to reinforcement learning could be promising to improve stability and convergence.