PatchAttack: A Black-box Texture-based Attack with Reinforcement Learning
This addresses the challenge of creating effective and query-efficient adversarial attacks for machine learning models, particularly in security-critical applications, though it is incremental as it builds on existing patch-based methods.
The paper tackles the problem of query inefficiency and poor targeted attack performance in black-box patch-based adversarial attacks by proposing PatchAttack, which uses reinforcement learning to optimize textured patches, achieving >99% success rate on ImageNet with only 3% image manipulation for non-targeted attacks and 10% on average for targeted attacks.
Patch-based attacks introduce a perceptible but localized change to the input that induces misclassification. A limitation of current patch-based black-box attacks is that they perform poorly for targeted attacks, and even for the less challenging non-targeted scenarios, they require a large number of queries. Our proposed PatchAttack is query efficient and can break models for both targeted and non-targeted attacks. PatchAttack induces misclassifications by superimposing small textured patches on the input image. We parametrize the appearance of these patches by a dictionary of class-specific textures. This texture dictionary is learned by clustering Gram matrices of feature activations from a VGG backbone. PatchAttack optimizes the position and texture parameters of each patch using reinforcement learning. Our experiments show that PatchAttack achieves > 99% success rate on ImageNet for a wide range of architectures, while only manipulating 3% of the image for non-targeted attacks and 10% on average for targeted attacks. Furthermore, we show that PatchAttack circumvents state-of-the-art adversarial defense methods successfully.