Safe Semantics, Unsafe Interpretations: Tackling Implicit Reasoning Safety in Large Vision-Language Models
This addresses a critical safety issue for users of multimodal AI systems, though it is incremental as it builds on existing safety concerns with a new dataset and method.
The paper tackles the problem of Implicit Reasoning Safety vulnerabilities in Large Vision-Language Models, where benign combined inputs trigger unsafe outputs due to flawed reasoning, and demonstrates that simple In-Context Learning with their new dataset significantly mitigates these threats.
Large Vision-Language Models face growing safety challenges with multimodal inputs. This paper introduces the concept of Implicit Reasoning Safety, a vulnerability in LVLMs. Benign combined inputs trigger unsafe LVLM outputs due to flawed or hidden reasoning. To showcase this, we developed Safe Semantics, Unsafe Interpretations, the first dataset for this critical issue. Our demonstrations show that even simple In-Context Learning with SSUI significantly mitigates these implicit multimodal threats, underscoring the urgent need to improve cross-modal implicit reasoning.