Solving Trojan Detection Competitions with Linear Weight Classification
This addresses a security issue for AI practitioners by providing an effective detection method, though it appears incremental as it builds on existing formulations.
The paper tackles the problem of detecting Trojan backdoors in neural networks without triggered data by training a binary classifier on pre-processed model weights, achieving strong performance across multiple benchmarks.
Neural networks can conceal malicious Trojan backdoors that allow a trigger to covertly change the model behavior. Detecting signs of these backdoors, particularly without access to any triggered data, is the subject of ongoing research and open challenges. In one common formulation of the problem, we are given a set of clean and poisoned models and need to predict whether a given test model is clean or poisoned. In this paper, we introduce a detector that works remarkably well across many of the existing datasets and domains. It is obtained by training a binary classifier on a large number of models' weights after performing a few different pre-processing steps including feature selection and standardization, reference model weights subtraction, and model alignment prior to detection. We evaluate this algorithm on a diverse set of Trojan detection benchmarks and domains and examine the cases where the approach is most and least effective.