Avoiding Copyright Infringement via Large Language Model Unlearning
This addresses copyright infringement risks for model owners in real-world scenarios, though it is incremental as it builds on existing unlearning methods.
The paper tackles the problem of sequential unlearning of copyrighted content from large language models to avoid legal risks, proposing Stable Sequential Unlearning (SSU) which achieves a trade-off between unlearning efficacy and retained language abilities, outperforming baselines.
Pre-trained Large Language Models (LLMs) have demonstrated remarkable capabilities but also pose risks by learning and generating copyrighted material, leading to significant legal and ethical concerns. In real-world scenarios, model owners need to continuously address copyright infringement as new requests for content removal emerge at different time points. This leads to the need for sequential unlearning, where copyrighted content is removed sequentially as new requests arise. Despite its practical relevance, sequential unlearning in the context of copyright infringement has not been rigorously explored in existing literature. To address this gap, we propose Stable Sequential Unlearning (SSU), a novel framework designed to unlearn copyrighted content from LLMs over multiple time steps. Our approach works by identifying and removing specific weight updates in the model's parameters that correspond to copyrighted content. We improve unlearning efficacy by introducing random labeling loss and ensuring the model retains its general-purpose knowledge by adjusting targeted parameters. Experimental results show that SSU achieves an effective trade-off between unlearning efficacy and general-purpose language abilities, outperforming existing baselines.