XIMAGENET-12: An Explainable AI Benchmark Dataset for Model Robustness Evaluation
This provides a benchmark for researchers to assess model robustness in challenging conditions, but it is incremental as it builds on existing datasets like ImageNet.
They tackled the challenge of evaluating visual model robustness for real-world applications by proposing XIMAGENET-12, an explainable dataset with over 200K images and 15,410 manual annotations, which includes 12 categories and six diverse scenarios to simulate practical conditions.
Despite the promising performance of existing visual models on public benchmarks, the critical assessment of their robustness for real-world applications remains an ongoing challenge. To bridge this gap, we propose an explainable visual dataset, XIMAGENET-12, to evaluate the robustness of visual models. XIMAGENET-12 consists of over 200K images with 15,410 manual semantic annotations. Specifically, we deliberately selected 12 categories from ImageNet, representing objects commonly encountered in practical life. To simulate real-world situations, we incorporated six diverse scenarios, such as overexposure, blurring, and color changes, etc. We further develop a quantitative criterion for robustness assessment, allowing for a nuanced understanding of how visual models perform under varying conditions, notably in relation to the background. We make the XIMAGENET-12 dataset and its corresponding code openly accessible at \url{https://sites.google.com/view/ximagenet-12/home}. We expect the introduction of the XIMAGENET-12 dataset will empower researchers to thoroughly evaluate the robustness of their visual models under challenging conditions.