MISA: Online Defense of Trojaned Models using Misattributions
This addresses the vulnerability of neural networks to Trojan attacks, providing a defense mechanism for security-critical applications, though it is incremental as it builds on existing attribution methods.
The paper tackles the problem of detecting Trojan triggers in neural networks at inference time by introducing MISA, an online defense method based on misattributions that analyzes feature attributions to identify anomalous activations, achieving 96% AUC across various benchmarks including recent trigger patterns with no known defenses.
Recent studies have shown that neural networks are vulnerable to Trojan attacks, where a network is trained to respond to specially crafted trigger patterns in the inputs in specific and potentially malicious ways. This paper proposes MISA, a new online approach to detect Trojan triggers for neural networks at inference time. Our approach is based on a novel notion called misattributions, which captures the anomalous manifestation of a Trojan activation in the feature space. Given an input image and the corresponding output prediction, our algorithm first computes the model's attribution on different features. It then statistically analyzes these attributions to ascertain the presence of a Trojan trigger. Across a set of benchmarks, we show that our method can effectively detect Trojan triggers for a wide variety of trigger patterns, including several recent ones for which there are no known defenses. Our method achieves 96% AUC for detecting images that include a Trojan trigger without any assumptions on the trigger pattern.