Multi-Attribute Open Set Recognition
This work addresses the need for explainable OSR in computer vision, but it is incremental as it builds on conventional OSR with simple extensions.
The paper tackles the problem of Open Set Recognition (OSR) by extending it to a multi-attribute setting, where unknown samples can be identified and categorized by their unknown visual attributes, and shows that existing baselines are vulnerable to spurious correlations, leading to poor out-of-distribution performance.
Open Set Recognition (OSR) extends image classification to an open-world setting, by simultaneously classifying known classes and identifying unknown ones. While conventional OSR approaches can detect Out-of-Distribution (OOD) samples, they cannot provide explanations indicating which underlying visual attribute(s) (e.g., shape, color or background) cause a specific sample to be unknown. In this work, we introduce a novel problem setup that generalizes conventional OSR to a multi-attribute setting, where multiple visual attributes are simultaneously recognized. Here, OOD samples can be not only identified but also categorized by their unknown attribute(s). We propose simple extensions of common OSR baselines to handle this novel scenario. We show that these baselines are vulnerable to shortcuts when spurious correlations exist in the training dataset. This leads to poor OOD performance which, according to our experiments, is mainly due to unintended cross-attribute correlations of the predicted confidence scores. We provide an empirical evidence showing that this behavior is consistent across different baselines on both synthetic and real world datasets.