To Know or Not To Know? Analyzing Self-Consistency of Large Language Models under Ambiguity
This addresses trustworthiness issues in LLMs for users relying on factual accuracy, though it is incremental as it focuses on a specific aspect of model behavior.
The paper tackles the problem of self-inconsistency in large language models (LLMs) when handling ambiguous entities, finding that LLMs achieve only 85% accuracy on average and as low as 75% with underspecified prompts, indicating struggles in applying factual knowledge consistently.
One of the major aspects contributing to the striking performance of large language models (LLMs) is the vast amount of factual knowledge accumulated during pre-training. Yet, many LLMs suffer from self-inconsistency, which raises doubts about their trustworthiness and reliability. This paper focuses on entity type ambiguity, analyzing the proficiency and consistency of state-of-the-art LLMs in applying factual knowledge when prompted with ambiguous entities. To do so, we propose an evaluation protocol that disentangles knowing from applying knowledge, and test state-of-the-art LLMs on 49 ambiguous entities. Our experiments reveal that LLMs struggle with choosing the correct entity reading, achieving an average accuracy of only 85%, and as low as 75% with underspecified prompts. The results also reveal systematic discrepancies in LLM behavior, showing that while the models may possess knowledge, they struggle to apply it consistently, exhibit biases toward preferred readings, and display self-inconsistencies. This highlights the need to address entity ambiguity in the future for more trustworthy LLMs.