Noise or Signal: The Role of Image Backgrounds in Object Recognition
This addresses the issue of spurious correlations in machine learning for researchers and practitioners, though it is incremental as it builds on existing analysis of model biases.
The study tackled the problem of object recognition models relying on image backgrounds, finding that models can achieve non-trivial accuracy using only backgrounds and misclassify images up to 87.5% of the time with adversarial backgrounds, with more accurate models depending less on backgrounds.
We assess the tendency of state-of-the-art object recognition models to depend on signals from image backgrounds. We create a toolkit for disentangling foreground and background signal on ImageNet images, and find that (a) models can achieve non-trivial accuracy by relying on the background alone, (b) models often misclassify images even in the presence of correctly classified foregrounds--up to 87.5% of the time with adversarially chosen backgrounds, and (c) more accurate models tend to depend on backgrounds less. Our analysis of backgrounds brings us closer to understanding which correlations machine learning models use, and how they determine models' out of distribution performance.