Err on the Side of Texture: Texture Bias on Real Data
This addresses texture bias in image classification models, which undermines accuracy and trustworthiness, by providing a novel metric for real-world settings, though it is incremental as it builds on existing bias measurement approaches.
The paper tackled the problem of texture bias in image classification models by introducing the Texture Association Value (TAV) metric, and found that texture bias explains over 90% of natural adversarial examples due to misaligned textures causing confident mispredictions.
Bias significantly undermines both the accuracy and trustworthiness of machine learning models. To date, one of the strongest biases observed in image classification models is texture bias-where models overly rely on texture information rather than shape information. Yet, existing approaches for measuring and mitigating texture bias have not been able to capture how textures impact model robustness in real-world settings. In this work, we introduce the Texture Association Value (TAV), a novel metric that quantifies how strongly models rely on the presence of specific textures when classifying objects. Leveraging TAV, we demonstrate that model accuracy and robustness are heavily influenced by texture. Our results show that texture bias explains the existence of natural adversarial examples, where over 90% of these samples contain textures that are misaligned with the learned texture of their true label, resulting in confident mispredictions.