Cyber for AI at SemEval-2025 Task 4: Forgotten but Not Lost: The Balancing Act of Selective Unlearning in Large Language Models
This work addresses the challenge of efficient selective unlearning in LLMs for privacy and compliance, but it is incremental as it builds on existing unlearning methods within a specific task.
This paper tackles the problem of selectively removing sensitive or obsolete data from large language models (LLMs) to address privacy, ethics, and compliance issues, achieving aggregate scores of 0.409 and 0.389 on test sets for 7B and 1B models, respectively, in verifiable unlearning.
Large Language Models (LLMs) face significant challenges in maintaining privacy, ethics, and compliance, when sensitive or obsolete data must be selectively removed. Retraining these models from scratch is computationally infeasible, necessitating efficient alternatives. As part of the SemEval 2025 Task 4, this work focuses on the application of selective unlearning in LLMs to address this challenge. In this paper, we present our experiments and findings, primarily leveraging global weight modification to achieve an equilibrium between effectiveness of unlearning, knowledge retention, and target model's post-unlearning utility. We also detail the task-specific evaluation mechanism, results, and challenges. Our algorithms have achieved an aggregate score of 0.409 and 0.389 on the test set for 7B and 1B target models, respectively, demonstrating promising results in verifiable LLM unlearning.