Distilling Model Failures as Directions in Latent Space
This work addresses the need for automated and scalable failure analysis in machine learning models, particularly for researchers and practitioners dealing with dataset biases and model robustness, though it builds incrementally on existing techniques like linear classifiers and diffusion models.
The paper tackles the problem of labor-intensive and dataset-specific methods for isolating hard subpopulations and spurious correlations by presenting a scalable method that automatically distills a model's failure modes as directions in latent space, enabling discovery of challenging subpopulations and generation of synthetic images for data augmentation.
Existing methods for isolating hard subpopulations and spurious correlations in datasets often require human intervention. This can make these methods labor-intensive and dataset-specific. To address these shortcomings, we present a scalable method for automatically distilling a model's failure modes. Specifically, we harness linear classifiers to identify consistent error patterns, and, in turn, induce a natural representation of these failure modes as directions within the feature space. We demonstrate that this framework allows us to discover and automatically caption challenging subpopulations within the training dataset. Moreover, by combining our framework with off-the-shelf diffusion models, we can generate images that are especially challenging for the analyzed model, and thus can be used to perform synthetic data augmentation that helps remedy the model's failure modes. Code available at https://github.com/MadryLab/failure-directions