ImageNet-trained CNNs are not biased towards texture: Revisiting feature reliance through controlled suppression
This challenges a widely held belief in deep learning about feature bias, with implications for model interpretation and design across multiple application domains.
The study revisited the hypothesis that CNNs are texture-biased by proposing a controlled suppression framework to quantify feature reliance, finding that CNNs primarily rely on local shape features, not texture, and that reliance patterns vary across domains like computer vision, medical imaging, and remote sensing.
The hypothesis that Convolutional Neural Networks (CNNs) are inherently texture-biased has shaped much of the discourse on feature use in deep learning. We revisit this hypothesis by examining limitations in the cue-conflict experiment by Geirhos et al. To address these limitations, we propose a domain-agnostic framework that quantifies feature reliance through systematic suppression of shape, texture, and color cues, avoiding the confounds of forced-choice conflicts. By evaluating humans and neural networks under controlled suppression conditions, we find that CNNs are not inherently texture-biased but predominantly rely on local shape features. Nonetheless, this reliance can be substantially mitigated through modern training strategies or architectures (ConvNeXt, ViTs). We further extend the analysis across computer vision, medical imaging, and remote sensing, revealing that reliance patterns differ systematically: computer vision models prioritize shape, medical imaging models emphasize color, and remote sensing models exhibit a stronger reliance on texture. Code is available at https://github.com/tomburgert/feature-reliance.