CREST: Universal Safety Guardrails Through Cluster-Guided Cross-Lingual Transfer
This addresses the safety deployment gap for global populations using low-resource languages, representing an incremental improvement in multilingual safety systems.
The paper tackles the problem of underrepresented safety guardrails for low-resource languages in large language models by introducing CREST, a parameter-efficient multilingual safety classification model that supports 100 languages with only 0.5B parameters and outperforms existing state-of-the-art guardrails of comparable scale.
Ensuring content safety in large language models (LLMs) is essential for their deployment in real-world applications. However, existing safety guardrails are predominantly tailored for high-resource languages, leaving a significant portion of the world's population underrepresented who communicate in low-resource languages. To address this, we introduce CREST (CRoss-lingual Efficient Safety Transfer), a parameter-efficient multilingual safety classification model that supports 100 languages with only 0.5B parameters. By training on a strategically chosen subset of only 13 high-resource languages, our model utilizes cluster-based cross-lingual transfer from a few to 100 languages, enabling effective generalization to both unseen high-resource and low-resource languages. This approach addresses the challenge of limited training data in low-resource settings. We conduct comprehensive evaluations across six safety benchmarks to demonstrate that CREST outperforms existing state-of-the-art guardrails of comparable scale and achieves competitive results against models with significantly larger parameter counts (2.5B parameters and above). Our findings highlight the limitations of language-specific guardrails and underscore the importance of developing universal, language-agnostic safety systems that can scale effectively to serve global populations.