Attention Hijacking in Trojan Transformers
This addresses security threats in AI systems by detecting Trojans in widely used Transformer models, though it is incremental as it builds on existing Trojan attack research.
The paper investigates Trojan attacks in Transformer models, revealing that trigger tokens hijack attention weights in both BERT and ViT models, and proposes an Attention-Hijacking Trojan Detector (AHTD) to identify these attacks.
Trojan attacks pose a severe threat to AI systems. Recent works on Transformer models received explosive popularity and the self-attentions are now indisputable. This raises a central question: Can we reveal the Trojans through attention mechanisms in BERTs and ViTs? In this paper, we investigate the attention hijacking pattern in Trojan AIs, \ie, the trigger token ``kidnaps'' the attention weights when a specific trigger is present. We observe the consistent attention hijacking pattern in Trojan Transformers from both Natural Language Processing (NLP) and Computer Vision (CV) domains. This intriguing property helps us to understand the Trojan mechanism in BERTs and ViTs. We also propose an Attention-Hijacking Trojan Detector (AHTD) to discriminate the Trojan AIs from the clean ones.