SafeCoT: Improving VLM Safety with Minimal Reasoning
This addresses safety alignment for vision-language models, particularly in critical applications, but appears incremental as it builds on existing chain-of-thought methods.
The paper tackles the problem of ensuring safe and appropriate responses from vision-language models in high-risk or ambiguous scenarios by introducing SafeCoT, a lightweight framework that uses rule-based chain-of-thought supervision to improve refusal behavior, resulting in significant reductions in overrefusal and enhanced generalization across multiple benchmarks.
Ensuring safe and appropriate responses from vision-language models (VLMs) remains a critical challenge, particularly in high-risk or ambiguous scenarios. We introduce SafeCoT, a lightweight, interpretable framework that leverages rule-based chain-of-thought (CoT) supervision to improve refusal behavior in VLMs. Unlike prior methods that rely on large-scale safety annotations or complex modeling, SafeCoT uses minimal supervision to help models reason about safety risks and make context-aware refusals. Experiments across multiple benchmarks show that SafeCoT significantly reduces overrefusal and enhances generalization, even with limited training data. Our approach offers a scalable solution for aligning VLMs with safety-critical objectives.