On the interplay of adversarial robustness and architecture components: patches, convolution and attention
This work addresses the limited understanding of architectural factors in adversarial robustness for image classification, providing incremental insights for researchers in machine learning security.
The study investigated how different architectural components, such as patches, convolution, and attention, affect adversarial robustness in image classifiers, finding that specific changes from ResNet to ConvNeXt led to nearly 10% higher robustness against adversarial attacks.
In recent years novel architecture components for image classification have been developed, starting with attention and patches used in transformers. While prior works have analyzed the influence of some aspects of architecture components on the robustness to adversarial attacks, in particular for vision transformers, the understanding of the main factors is still limited. We compare several (non)-robust classifiers with different architectures and study their properties, including the effect of adversarial training on the interpretability of the learnt features and robustness to unseen threat models. An ablation from ResNet to ConvNeXt reveals key architectural changes leading to almost $10\%$ higher $\ell_\infty$-robustness.