Why Adversarial Reprogramming Works, When It Fails, and How to Tell the Difference
This work addresses a gap in explaining the factors behind adversarial reprogramming, which is important for security and transfer learning applications, though it is incremental in providing a theoretical model.
The paper tackled the problem of understanding when adversarial reprogramming of machine-learning models succeeds or fails, finding that success depends on the size of the average input gradient, which increases with gradient alignment and input dimensionality, as validated in fourteen reprogramming tasks.
Adversarial reprogramming allows repurposing a machine-learning model to perform a different task. For example, a model trained to recognize animals can be reprogrammed to recognize digits by embedding an adversarial program in the digit images provided as input. Recent work has shown that adversarial reprogramming may not only be used to abuse machine-learning models provided as a service, but also beneficially, to improve transfer learning when training data is scarce. However, the factors affecting its success are still largely unexplained. In this work, we develop a first-order linear model of adversarial reprogramming to show that its success inherently depends on the size of the average input gradient, which grows when input gradients are more aligned, and when inputs have higher dimensionality. The results of our experimental analysis, involving fourteen distinct reprogramming tasks, show that the above factors are correlated with the success and the failure of adversarial reprogramming.