FanarGuard: A Culturally-Aware Moderation Filter for Arabic Language Models
This addresses the need for culturally-aware moderation in Arabic language models, offering a practical improvement over existing filters that are culturally narrow.
The authors tackled the problem of content moderation filters overlooking cultural context by introducing FanarGuard, a bilingual filter for Arabic and English that evaluates safety and cultural alignment, achieving stronger agreement with human annotations than inter-annotator reliability while matching state-of-the-art safety performance.
Content moderation filters are a critical safeguard against alignment failures in language models. Yet most existing filters focus narrowly on general safety and overlook cultural context. In this work, we introduce FanarGuard, a bilingual moderation filter that evaluates both safety and cultural alignment in Arabic and English. We construct a dataset of over 468K prompt and response pairs, drawn from synthetic and public datasets, scored by a panel of LLM judges on harmlessness and cultural awareness, and use it to train two filter variants. To rigorously evaluate cultural alignment, we further develop the first benchmark targeting Arabic cultural contexts, comprising over 1k norm-sensitive prompts with LLM-generated responses annotated by human raters. Results show that FanarGuard achieves stronger agreement with human annotations than inter-annotator reliability, while matching the performance of state-of-the-art filters on safety benchmarks. These findings highlight the importance of integrating cultural awareness into moderation and establish FanarGuard as a practical step toward more context-sensitive safeguards.