SOOD-ImageNet: a Large-Scale Dataset for Semantic Out-Of-Distribution Image Classification and Semantic Segmentation
This addresses the problem of limited and semantically inadequate OOD benchmarks for computer vision researchers, providing a scalable dataset to improve model generalizability in real-world scenarios.
The authors tackled the limitations of existing Out-of-Distribution (OOD) benchmarks by introducing SOOD-ImageNet, a large-scale dataset with around 1.6M images across 56 classes, designed for semantic shift challenges in image classification and semantic segmentation, and demonstrated its potential to advance OOD research through extensive model evaluations.
Out-of-Distribution (OOD) detection in computer vision is a crucial research area, with related benchmarks playing a vital role in assessing the generalizability of models and their applicability in real-world scenarios. However, existing OOD benchmarks in the literature suffer from two main limitations: (1) they often overlook semantic shift as a potential challenge, and (2) their scale is limited compared to the large datasets used to train modern models. To address these gaps, we introduce SOOD-ImageNet, a novel dataset comprising around 1.6M images across 56 classes, designed for common computer vision tasks such as image classification and semantic segmentation under OOD conditions, with a particular focus on the issue of semantic shift. We ensured the necessary scalability and quality by developing an innovative data engine that leverages the capabilities of modern vision-language models, complemented by accurate human checks. Through extensive training and evaluation of various models on SOOD-ImageNet, we showcase its potential to significantly advance OOD research in computer vision. The project page is available at https://github.com/bach05/SOODImageNet.git.