OutSafe-Bench: A Benchmark for Multimodal Offensive Content Detection in Large Language Models
This addresses the need for robust safety evaluation in MLLMs, which are increasingly integrated into everyday tools, by providing a more comprehensive benchmark than existing limited ones, though it is incremental in improving evaluation methods.
The authors tackled the problem of evaluating safety vulnerabilities in multimodal large language models (MLLMs) by introducing OutSafe-Bench, a comprehensive benchmark with a large-scale dataset spanning four modalities and nine risk categories, and found that nine state-of-the-art MLLMs exhibit persistent and substantial safety vulnerabilities.
Since Multimodal Large Language Models (MLLMs) are increasingly being integrated into everyday tools and intelligent agents, growing concerns have arisen regarding their possible output of unsafe contents, ranging from toxic language and biased imagery to privacy violations and harmful misinformation. Current safety benchmarks remain highly limited in both modality coverage and performance evaluations, often neglecting the extensive landscape of content safety. In this work, we introduce OutSafe-Bench, the first most comprehensive content safety evaluation test suite designed for the multimodal era. OutSafe-Bench includes a large-scale dataset that spans four modalities, featuring over 18,000 bilingual (Chinese and English) text prompts, 4,500 images, 450 audio clips and 450 videos, all systematically annotated across nine critical content risk categories. In addition to the dataset, we introduce a Multidimensional Cross Risk Score (MCRS), a novel metric designed to model and assess overlapping and correlated content risks across different categories. To ensure fair and robust evaluation, we propose FairScore, an explainable automated multi-reviewer weighted aggregation framework. FairScore selects top-performing models as adaptive juries, thereby mitigating biases from single-model judgments and enhancing overall evaluation reliability. Our evaluation of nine state-of-the-art MLLMs reveals persistent and substantial safety vulnerabilities, underscoring the pressing need for robust safeguards in MLLMs.