Fix your downsampling ASAP! Be natively more robust via Aliasing and Spectral Artifact free Pooling
This addresses a foundational issue in CNN design for computer vision, potentially enhancing robustness across various applications, though it is an incremental improvement over prior work on aliasing.
The paper tackles the problem of aliasing artifacts in CNN downsampling, which violates signal processing laws and correlates with vulnerability to adversarial attacks and distribution shifts, by proposing alias-free downsampling methods (FLC Pooling and ASAP) that improve robustness against common corruptions and adversarial attacks while maintaining similar clean accuracy on datasets like ImageNet-1k and CIFAR.
Convolutional Neural Networks (CNNs) are successful in various computer vision tasks. From an image and signal processing point of view, this success is counter-intuitive, as the inherent spatial pyramid design of most CNNs is apparently violating basic signal processing laws, i.e. the Sampling Theorem in their downsampling operations. This issue has been broadly neglected until recent work in the context of adversarial attacks and distribution shifts showed that there is a strong correlation between the vulnerability of CNNs and aliasing artifacts induced by bandlimit-violating downsampling. As a remedy, we propose an alias-free downsampling operation in the frequency domain, denoted Frequency Low Cut Pooling (FLC Pooling) which we further extend to Aliasing and Sinc Artifact-free Pooling (ASAP). ASAP is alias-free and removes further artifacts from sinc-interpolation. Our experimental evaluation on ImageNet-1k, ImageNet-C and CIFAR datasets on various CNN architectures demonstrates that networks using FLC Pooling and ASAP as downsampling methods learn more stable features as measured by their robustness against common corruptions and adversarial attacks, while maintaining a clean accuracy similar to the respective baseline models.