TOP: Backdoor Detection in Neural Networks via Transferability of Perturbation
This addresses a critical security issue for users of deep learning models by providing a method to identify compromised models without prior knowledge of the attack.
The paper tackled the problem of detecting backdoor poisoning attacks in deep neural networks without needing training data or trigger examples, by discovering that adversarial perturbations transfer more easily in poisoned models, and demonstrated detection on the TrojAI benchmark and other models.
Deep neural networks (DNNs) are vulnerable to "backdoor" poisoning attacks, in which an adversary implants a secret trigger into an otherwise normally functioning model. Detection of backdoors in trained models without access to the training data or example triggers is an important open problem. In this paper, we identify an interesting property of these models: adversarial perturbations transfer from image to image more readily in poisoned models than in clean models. This holds for a variety of model and trigger types, including triggers that are not linearly separable from clean data. We use this feature to detect poisoned models in the TrojAI benchmark, as well as additional models.