UDIS: Unsupervised Discovery of Bias in Deep Visual Recognition Models
It addresses the issue of bias in visual recognition models for researchers and practitioners, offering a scalable alternative to expensive crowdsourced annotations, though it is incremental as it builds on existing clustering and visualization techniques.
The paper tackles the problem of deep learning models learning spurious correlations that cause systematic failures for certain subpopulations, proposing UDIS, an unsupervised algorithm that identifies failure modes via hierarchical clustering and visualization, showing effectiveness on CelebA and MSCOCO datasets.
Deep learning models have been shown to learn spurious correlations from data that sometimes lead to systematic failures for certain subpopulations. Prior work has typically diagnosed this by crowdsourcing annotations for various protected attributes and measuring performance, which is both expensive to acquire and difficult to scale. In this work, we propose UDIS, an unsupervised algorithm for surfacing and analyzing such failure modes. UDIS identifies subpopulations via hierarchical clustering of dataset embeddings and surfaces systematic failure modes by visualizing low performing clusters along with their gradient-weighted class-activation maps. We show the effectiveness of UDIS in identifying failure modes in models trained for image classification on the CelebA and MSCOCO datasets.