RLCP: A Reinforcement Learning-based Copyright Protection Method for Text-to-Image Diffusion Model
This addresses copyright protection for users and developers of text-to-image models, but it is incremental as it builds on existing reinforcement learning frameworks.
The authors tackled the problem of copyright infringement in text-to-image diffusion models by proposing RLCP, a reinforcement learning-based method that reduces copyright-infringing content generation while preserving image quality, achieving significant risk reduction in experiments on mixed datasets.
The increasing sophistication of text-to-image generative models has led to complex challenges in defining and enforcing copyright infringement criteria and protection. Existing methods, such as watermarking and dataset deduplication, fail to provide comprehensive solutions due to the lack of standardized metrics and the inherent complexity of addressing copyright infringement in diffusion models. To deal with these challenges, we propose a Reinforcement Learning-based Copyright Protection(RLCP) method for Text-to-Image Diffusion Model, which minimizes the generation of copyright-infringing content while maintaining the quality of the model-generated dataset. Our approach begins with the introduction of a novel copyright metric grounded in copyright law and court precedents on infringement. We then utilize the Denoising Diffusion Policy Optimization (DDPO) framework to guide the model through a multi-step decision-making process, optimizing it using a reward function that incorporates our proposed copyright metric. Additionally, we employ KL divergence as a regularization term to mitigate some failure modes and stabilize RL fine-tuning. Experiments conducted on 3 mixed datasets of copyright and non-copyright images demonstrate that our approach significantly reduces copyright infringement risk while maintaining image quality.