LLM Judges Inconsistently Disagree Across Safety Criteria and Harm Categories
This paper addresses the problem of inconsistent safety evaluations by LLM judges, which is critical for practitioners relying on automated content moderation.
This paper evaluates the consistency of LLM judges in multi-dimensional, reference-free safety evaluations. It finds that LLMs are unreliable in identifying safety issues in regulated domains like finance but more reliable for overt harmful content like violence. The inconsistency varies significantly by safety criteria, language, and linguistic style, with high disagreement among different judges across domains, criteria, and languages.
We evaluate the consistency of automated judges in conducting a multi-dimensional safety evaluation in a reference-free setup. Our results indicate that Large Language Models are unreliable judges in identifying safety issues related to machine-generated advice in regulated domains such as finance, although they are more reliable at identifying more overt forms of unsafe/harmful content such as violence. The degree of inconsistency in a model's judgments can vary significantly by the chosen safety criteria and can be impacted by the language of the content and its linguistic style as well. Finally, there is high disagreement among different judges for the same output, across domains, safety criteria, and languages. These findings provide new insights on the practice of using LLMs as evaluators and offer several recommendations for practitioners on how to use automated judges in practical scenarios.