Mass-Producing Failures of Multimodal Systems with Language Models
This addresses the challenge of pre-deployment evaluation for multimodal AI systems, offering a tool to uncover critical failures that could impact applications like image generation and self-driving cars, though it is incremental as it builds on existing evaluation methods.
The paper tackles the problem of identifying unforeseen failures in multimodal systems by introducing MultiMon, which automatically discovers systematic failure patterns using language models, finding 14 such failures in CLIP that affect hundreds of inputs and propagate to systems like Midjourney and DALL-E.
Deployed multimodal systems can fail in ways that evaluators did not anticipate. In order to find these failures before deployment, we introduce MultiMon, a system that automatically identifies systematic failures -- generalizable, natural-language descriptions of patterns of model failures. To uncover systematic failures, MultiMon scrapes a corpus for examples of erroneous agreement: inputs that produce the same output, but should not. It then prompts a language model (e.g., GPT-4) to find systematic patterns of failure and describe them in natural language. We use MultiMon to find 14 systematic failures (e.g., "ignores quantifiers") of the CLIP text-encoder, each comprising hundreds of distinct inputs (e.g., "a shelf with a few/many books"). Because CLIP is the backbone for most state-of-the-art multimodal systems, these inputs produce failures in Midjourney 5.1, DALL-E, VideoFusion, and others. MultiMon can also steer towards failures relevant to specific use cases, such as self-driving cars. We see MultiMon as a step towards evaluation that autonomously explores the long tail of potential system failures. Code for MULTIMON is available at https://github.com/tsb0601/MultiMon.