Conservative Bias in Large Language Models: Measuring Relation Predictions
This addresses a critical issue for NLP practitioners and researchers by highlighting a trade-off in model reliability that can impact downstream applications, though it is incremental in nature.
The paper tackles the problem of conservative bias in large language models during relation extraction, where models frequently default to uninformative labels, leading to significant information loss. The study finds that conservative bias occurs twice as often as hallucination, using semantic similarity methods to quantify this effect.
Large language models (LLMs) exhibit pronounced conservative bias in relation extraction tasks, frequently defaulting to No_Relation label when an appropriate option is unavailable. While this behavior helps prevent incorrect relation assignments, our analysis reveals that it also leads to significant information loss when reasoning is not explicitly included in the output. We systematically evaluate this trade-off across multiple prompts, datasets, and relation types, introducing the concept of Hobson's choice to capture scenarios where models opt for safe but uninformative labels over hallucinated ones. Our findings suggest that conservative bias occurs twice as often as hallucination. To quantify this effect, we use SBERT and LLM prompts to capture the semantic similarity between conservative bias behaviors in constrained prompts and labels generated from semi-constrained and open-ended prompts.