Detecting Backdoors in Pre-trained Encoders
This addresses a crucial vulnerability in self-supervised learning for computer vision, where backdoors in encoders can propagate to downstream classifiers, and is novel as it extends detection to pre-trained encoders without labels.
The paper tackles the problem of detecting backdoor attacks in pre-trained encoders used in self-supervised learning, proposing DECREE as the first method that works without classifier headers or input labels, achieving high detection accuracy on over 400 trojaned encoders.
Self-supervised learning in computer vision trains on unlabeled data, such as images or (image, text) pairs, to obtain an image encoder that learns high-quality embeddings for input data. Emerging backdoor attacks towards encoders expose crucial vulnerabilities of self-supervised learning, since downstream classifiers (even further trained on clean data) may inherit backdoor behaviors from encoders. Existing backdoor detection methods mainly focus on supervised learning settings and cannot handle pre-trained encoders especially when input labels are not available. In this paper, we propose DECREE, the first backdoor detection approach for pre-trained encoders, requiring neither classifier headers nor input labels. We evaluate DECREE on over 400 encoders trojaned under 3 paradigms. We show the effectiveness of our method on image encoders pre-trained on ImageNet and OpenAI's CLIP 400 million image-text pairs. Our method consistently has a high detection accuracy even if we have only limited or no access to the pre-training dataset.