Analyzing Multi-Head Attention on Trojan BERT Models
This addresses security vulnerabilities in AI models for sentiment analysis, but it is incremental as it builds on existing trojan attack research.
The study analyzed multi-head attention in Transformer models to compare benign and trojan models for sentiment analysis, finding specific 'trojan' heads that cause misclassifications with triggers.
This project investigates the behavior of multi-head attention in Transformer models, specifically focusing on the differences between benign and trojan models in the context of sentiment analysis. Trojan attacks cause models to perform normally on clean inputs but exhibit misclassifications when presented with inputs containing predefined triggers. We characterize attention head functions in trojan and benign models, identifying specific 'trojan' heads and analyzing their behavior.