Direct Token Optimization: A Self-contained Approach to Large Language Model Unlearning
This addresses the need for practical and privacy-preserving unlearning in LLMs for applications like privacy protection and content moderation, offering a novel approach that is more efficient than existing methods.
The paper tackles the problem of machine unlearning for large language models by proposing Direct Token Optimization (DTO), a self-contained method that directly optimizes token-level objectives without external resources, achieving up to 16.8× improvement in forget quality on benchmark datasets while maintaining model utility.
Machine unlearning is an emerging technique that removes the influence of a subset of training data (forget set) from a model without full retraining, with applications including privacy protection, content moderation, and model correction. The key challenge lies in ensuring that the model completely forgets the knowledge of the forget set without compromising its overall utility. Existing unlearning methods for large language models (LLMs) often utilize auxiliary language models, retain datasets, or even commercial AI services for effective unlearning and maintaining the model utility. However, dependence on these external resources is often impractical and could potentially introduce additional privacy risks. In this work, we propose direct token optimization (DTO), a novel self-contained unlearning approach for LLMs that directly optimizes the token level objectives and eliminates the need for external resources. Given a sequence to unlearn, we identify two categories of tokens: target tokens, which capture critical knowledge for unlearning, and the remaining non-target tokens, which are crucial for maintaining the model utility. The former are used to optimize the unlearning objective, while the latter serve to preserve the model's performance. The experimental results show that the proposed DTO achieves up to 16.8$\times$ improvement in forget quality on several benchmark datasets than the latest baselines while maintaining a comparable level of model utility.