A review of faithfulness metrics for hallucination assessment in Large Language Models
It addresses the problem of unfaithful responses in LLMs, which pose risks to their widespread use, by reviewing evaluation methods to enhance trust, but it is incremental as it synthesizes existing research without introducing new metrics.
This review examines faithfulness metrics for hallucination assessment in Large Language Models (LLMs) across tasks like summarization and question-answering, finding that using LLMs as evaluators correlates best with human judgment and that methods like retrieval augmented generation improve faithfulness.
This review examines the means with which faithfulness has been evaluated across open-ended summarization, question-answering and machine translation tasks. We find that the use of LLMs as a faithfulness evaluator is commonly the metric that is most highly correlated with human judgement. The means with which other studies have mitigated hallucinations is discussed, with both retrieval augmented generation (RAG) and prompting framework approaches having been linked with superior faithfulness, whilst other recommendations for mitigation are provided. Research into faithfulness is integral to the continued widespread use of LLMs, as unfaithful responses can pose major risks to many areas whereby LLMs would otherwise be suitable. Furthermore, evaluating open-ended generation provides a more comprehensive measure of LLM performance than commonly used multiple-choice benchmarking, which can help in advancing the trust that can be placed within LLMs.