The Problem with Safety Classification is not just the Models
This work highlights problems in safety classification for LLMs, particularly in multilingual contexts, which is incremental as it builds on existing safety testing research.
The paper demonstrates that multilingual disparities exist in 5 safety classification models across 18 languages and identifies issues with evaluation datasets, arguing that shortcomings are not solely due to the models.
Studying the robustness of Large Language Models (LLMs) to unsafe behaviors is an important topic of research today. Building safety classification models or guard models, which are fine-tuned models for input/output safety classification for LLMs, is seen as one of the solutions to address the issue. Although there is a lot of research on the safety testing of LLMs themselves, there is little research on evaluating the effectiveness of such safety classifiers or the evaluation datasets used for testing them, especially in multilingual scenarios. In this position paper, we demonstrate how multilingual disparities exist in 5 safety classification models by considering datasets covering 18 languages. At the same time, we identify potential issues with the evaluation datasets, arguing that the shortcomings of current safety classifiers are not only because of the models themselves. We expect that these findings will contribute to the discussion on developing better methods to identify harmful content in LLM inputs across languages.