Benchmarking Adversarial Patch Selection and Location
This addresses the reliability of modern vision models against adversarial attacks, with incremental improvements in attack methods.
The paper tackled the problem of adversarial patch attacks on vision models by creating PatchMap, a spatially exhaustive benchmark of patch placement, which revealed systematic hot-spots where small patches cause confident misclassifications and large drops in model confidence. The result was a simple segmentation guided placement heuristic that boosted attack success rates by 8 to 13 percentage points across five architectures compared to random or fixed placements.
Adversarial patch attacks threaten the reliability of modern vision models. We present PatchMap, the first spatially exhaustive benchmark of patch placement, built by evaluating over 1.5e8 forward passes on ImageNet validation images. PatchMap reveals systematic hot-spots where small patches (as little as 2% of the image) induce confident misclassifications and large drops in model confidence. To demonstrate its utility, we propose a simple segmentation guided placement heuristic that leverages off the shelf masks to identify vulnerable regions without any gradient queries. Across five architectures-including adversarially trained ResNet50, our method boosts attack success rates by 8 to 13 percentage points compared to random or fixed placements. We publicly release PatchMap and the code implementation. The full PatchMap bench (6.5B predictions, multiple backbones) will be released soon to further accelerate research on location-aware defenses and adaptive attacks.