Erasure or Erosion? Evaluating Compositional Degradation in Unlearned Text-To-Image Diffusion Models
This work addresses the problem of balancing erasure and generative integrity in unlearning for AI safety, highlighting limitations in current evaluation practices for researchers and practitioners.
The study systematically evaluated the impact of concept unlearning methods on text-to-image diffusion models, revealing a trade-off where methods achieving strong erasure often degrade compositional capabilities like attribute binding and spatial reasoning, while those preserving composition fail to provide robust erasure.
Post-hoc unlearning has emerged as a practical mechanism for removing undesirable concepts from large text-to-image diffusion models. However, prior work primarily evaluates unlearning through erasure success; its impact on broader generative capabilities remains poorly understood. In this work, we conduct a systematic empirical study of concept unlearning through the lens of compositional text-to-image generation. Focusing on nudity removal in Stable Diffusion 1.4, we evaluate a diverse set of state-of-the-art unlearning methods using T2I-CompBench++ and GenEval, alongside established unlearning benchmarks. Our results reveal a consistent trade-off between unlearning effectiveness and compositional integrity: methods that achieve strong erasure frequently incur substantial degradation in attribute binding, spatial reasoning, and counting. Conversely, approaches that preserve compositional structure often fail to provide robust erasure. These findings highlight limitations of current evaluation practices and underscore the need for unlearning objectives that explicitly account for semantic preservation beyond targeted suppression.