Feature-Space Adversarial Robustness Certification for Multimodal Large Language Models
This addresses the problem of adversarial robustness for users of MLLMs in applications like vision-language tasks, though it is incremental as it builds on existing smoothing and certification techniques.
The paper tackles the vulnerability of multimodal large language models (MLLMs) to adversarial perturbations by proposing Feature-space Smoothing (FS), a framework that provides certified robustness guarantees at the feature representation level, resulting in strong certified feature-space robustness and robust task-oriented performance across diverse applications.
Multimodal large language models (MLLMs) exhibit strong capabilities across diverse applications, yet remain vulnerable to adversarial perturbations that distort their feature representations and induce erroneous predictions. To address this vulnerability, we propose Feature-space Smoothing (FS), a general framework that provides certified robustness guarantees at the feature representation level of MLLMs. We theoretically prove that FS converts a given feature extractor into a smoothed variant that is guaranteed a certified lower bound on the cosine similarity between clean and adversarial features under $\ell_2$-bounded perturbations. Moreover, we establish that the value of this Feature Cosine Similarity Bound (FCSB) is determined by the intrinsic Gaussian robustness score of the given encoder. Building on this insight, we introduce the Gaussian Smoothness Booster (GSB), a plug-and-play module that enhances the Gaussian robustness score of pretrained MLLMs, thereby strengthening the robustness guaranteed by FS, without requiring additional MLLM retraining. Extensive experiments demonstrate that applying the FS to various MLLMs yields strong certified feature-space robustness and consistently leads to robust task-oriented performance across diverse applications.