TEMPO-Diffusion: Temporally Exposed Malicious Poisoning of Diffusion Models
This work addresses the need for more realistic and stealthy backdoor attacks on diffusion models, which is important for security researchers studying the risks of using synthetic training data.
TEMPO-Diffusion introduces a targeted backdoor attack on diffusion models that localizes malicious behavior to a temporal, in-distribution exposure, achieving high attack success rates on downstream classifiers across CIFAR10, GTSRB, and a new traffic-sign dataset.
Noise-based backdoor attacks on diffusion models typically rely on input-time trigger injection, untargeted activation, and out-of-distribution target generation. Such assumptions reduce both the stealthiness and the practical relevance of these attacks. In this work, we present TEMPO-Diffusion, a targeted backdoor framework that localizes the malicious distribution shift to a temporal, in-distribution exposure. TEMPO-Diffusion supports: (i) targeted attacks on and to specific classes, (ii) multiple sub-image backdoors that reconstruct specific features within multiple, different output images and at multiple locations, and (iii) in-painting with time-conditioned triggers. To study relevant, practical security concerns in leveraging backdoored diffusion models for synthetic training data, we also introduce CALISA: a balanced, region-aware traffic-sign dataset emphasizing Canadian and U.S. road signs. Across CIFAR10, GTSRB, and CALISA, our experiments show that TEMPO-Diffusion can reliably poison class-specific synthetic data generation and induce high attack success rates in downstream classifiers trained on that data.