SugarCrepe: Fixing Hackable Benchmarks for Vision-Language Compositionality
This addresses the problem of hackable benchmarks for researchers in vision-language AI, exposing overestimated progress and enabling more reliable evaluation.
The authors identified significant biases in existing vision-language compositionality benchmarks, which allowed blind models to outperform state-of-the-art models, and introduced SugarCrepe, a new benchmark using large language models and adversarial refinement to reduce biases, revealing that previous improvements were overestimated.
In the last year alone, a surge of new benchmarks to measure compositional understanding of vision-language models have permeated the machine learning ecosystem. Given an image, these benchmarks probe a model's ability to identify its associated caption amongst a set of compositional distractors. Surprisingly, we find significant biases in all these benchmarks rendering them hackable. This hackability is so dire that blind models with no access to the image outperform state-of-the-art vision-language models. To remedy this rampant vulnerability, we introduce SugarCrepe, a new benchmark for vision-language compositionality evaluation. We employ large language models, instead of rule-based templates used in previous benchmarks, to generate fluent and sensical hard negatives, and utilize an adversarial refinement mechanism to maximally reduce biases. We re-evaluate state-of-the-art models and recently proposed compositionality inducing strategies, and find that their improvements were hugely overestimated, suggesting that more innovation is needed in this important direction. We release SugarCrepe and the code for evaluation at: https://github.com/RAIVNLab/sugar-crepe.