Contextual Image Attack: How Visual Context Exposes Multimodal Safety Vulnerabilities
This work addresses safety risks for users of advanced MLLMs by exposing vulnerabilities through visual context, representing a novel attack method rather than an incremental improvement.
The paper tackles the problem of safety vulnerabilities in Multimodal Large Language Models (MLLMs) by proposing Contextual Image Attack (CIA), an image-centric method that embeds harmful queries into visual contexts, achieving high toxicity scores of 4.73 and 4.83 and Attack Success Rates up to 91.07% against models like GPT-4o and Qwen2.5-VL-72B.
While Multimodal Large Language Models (MLLMs) show remarkable capabilities, their safety alignments are susceptible to jailbreak attacks. Existing attack methods typically focus on text-image interplay, treating the visual modality as a secondary prompt. This approach underutilizes the unique potential of images to carry complex, contextual information. To address this gap, we propose a new image-centric attack method, Contextual Image Attack (CIA), which employs a multi-agent system to subtly embeds harmful queries into seemingly benign visual contexts using four distinct visualization strategies. To further enhance the attack's efficacy, the system incorporate contextual element enhancement and automatic toxicity obfuscation techniques. Experimental results on the MMSafetyBench-tiny dataset show that CIA achieves high toxicity scores of 4.73 and 4.83 against the GPT-4o and Qwen2.5-VL-72B models, respectively, with Attack Success Rates (ASR) reaching 86.31\% and 91.07\%. Our method significantly outperforms prior work, demonstrating that the visual modality itself is a potent vector for jailbreaking advanced MLLMs.