SCAM: A Real-World Typographic Robustness Evaluation for Multimodal Foundation Models
This addresses the robustness issue in multimodal AI systems for researchers and developers, but it is incremental as it focuses on dataset creation and benchmarking.
The paper tackles the problem of typographic attacks degrading multimodal foundation models by introducing SCAM, the largest and diverse real-world dataset of such attacks, and finds that these attacks significantly reduce performance, with state-of-the-art models showing vulnerabilities, though larger language model backbones help mitigate this.
Typographic attacks exploit the interplay between text and visual content in multimodal foundation models, causing misclassifications when misleading text is embedded within images. Existing datasets are limited in size and diversity, making it difficult to study such vulnerabilities. In this paper, we introduce SCAM, the largest and most diverse dataset of real-world typographic attack images to date, containing 1162 images across hundreds of object categories and attack words. Through extensive benchmarking of Vision-Language Models on SCAM, we demonstrate that typographic attacks significantly degrade performance, and identify that training data and model architecture influence the susceptibility to these attacks. Our findings indicate that typographic attacks remain effective against state-of-the-art Large Vision-Language Models, especially those employing vision encoders inherently vulnerable to such attacks. However, employing larger Large Language Model backbones reduces this vulnerability while simultaneously enhancing typographic understanding. Additionally, we demonstrate that synthetic attacks closely resemble real-world (handwritten) attacks, validating their use in research. Our work provides a comprehensive resource and empirical insights to facilitate future research toward robust and trustworthy multimodal AI systems. Finally, we publicly release the datasets introduced in this paper, along with the code for evaluations under www.bliss.berlin/research/scam.