Robust Context-Aware Object Recognition
This addresses robustness issues in object recognition for real-world deployment, offering a novel solution to a known bottleneck.
The paper tackles the problem of shortcut learning in visual recognition, where models over-rely on background context, by proposing RCOR, a method that jointly achieves robustness and context-awareness without compromising either, improving performance on in-domain and out-of-domain datasets without fine-tuning.
In visual recognition, both the object of interest (referred to as foreground, FG, for simplicity) and its surrounding context (background, BG) play an important role. However, standard supervised learning often leads to unintended over-reliance on the BG, known as shortcut learning of spurious correlations, limiting model robustness in real-world deployment settings. In the literature, the problem is mainly addressed by suppressing the BG, sacrificing context information for improved generalization. We propose RCOR -- Robust Context-Aware Object Recognition -- the first approach that jointly achieves robustness and context-awareness without compromising either. RCOR treats localization as an integral part of recognition to decouple object-centric and context-aware modelling, followed by a robust, non-parametric fusion. It improves the performance of both supervised models and VLM on datasets with both in-domain and out-of-domain BG, even without fine-tuning. The results confirm that localization before recognition is now possible even in complex scenes as in ImageNet-1k.