Give Me Your Attention: Dot-Product Attention Considered Harmful for Adversarial Patch Robustness
This exposes a critical security flaw in widely used attention-based models, impacting applications in image recognition and object detection where robustness is essential.
The paper identifies a vulnerability in dot-product attention mechanisms used in vision transformers, where adversarial patches can misdirect attention and drastically reduce model performance. It demonstrates that patches covering just 0.5% of the input can drop ViT accuracy on ImageNet to 0% and reduce DETR mAP on MS COCO to under 3%.
Neural architectures based on attention such as vision transformers are revolutionizing image recognition. Their main benefit is that attention allows reasoning about all parts of a scene jointly. In this paper, we show how the global reasoning of (scaled) dot-product attention can be the source of a major vulnerability when confronted with adversarial patch attacks. We provide a theoretical understanding of this vulnerability and relate it to an adversary's ability to misdirect the attention of all queries to a single key token under the control of the adversarial patch. We propose novel adversarial objectives for crafting adversarial patches which target this vulnerability explicitly. We show the effectiveness of the proposed patch attacks on popular image classification (ViTs and DeiTs) and object detection models (DETR). We find that adversarial patches occupying 0.5% of the input can lead to robust accuracies as low as 0% for ViT on ImageNet, and reduce the mAP of DETR on MS COCO to less than 3%.