Large Language Models are Inconsistent and Biased Evaluators
This addresses the robustness issue of LLM evaluators for NLP researchers, but it is incremental as it builds on existing work to mitigate known limitations.
The paper tackles the problem of LLMs being inconsistent and biased evaluators in NLP tasks, confirming biases like familiarity preference and low inter-sample agreement, and proposes recipes that improve performance over state-of-the-art LLM evaluators on the RoSE dataset.
The zero-shot capability of Large Language Models (LLMs) has enabled highly flexible, reference-free metrics for various tasks, making LLM evaluators common tools in NLP. However, the robustness of these LLM evaluators remains relatively understudied; existing work mainly pursued optimal performance in terms of correlating LLM scores with human expert scores. In this paper, we conduct a series of analyses using the SummEval dataset and confirm that LLMs are biased evaluators as they: (1) exhibit familiarity bias-a preference for text with lower perplexity, (2) show skewed and biased distributions of ratings, and (3) experience anchoring effects for multi-attribute judgments. We also found that LLMs are inconsistent evaluators, showing low "inter-sample" agreement and sensitivity to prompt differences that are insignificant to human understanding of text quality. Furthermore, we share recipes for configuring LLM evaluators to mitigate these limitations. Experimental results on the RoSE dataset demonstrate improvements over the state-of-the-art LLM evaluators.