Semantic Faithfulness and Entropy Production Measures to Tame Your LLM Demons and Manage Hallucinations
This work addresses the challenge of managing hallucinations in LLMs for tasks like document summarization, though it appears incremental as it builds on existing evaluation methods with new metrics.
The paper tackles the problem of evaluating faithfulness in Large Language Models (LLMs) by proposing two unsupervised metrics: semantic faithfulness (SF) and semantic entropy production (SEP), which quantify alignment between context and answers using information theory and thermodynamics, and demonstrates their application on LLM summarization of SEC 10-K filings.
Evaluating faithfulness of Large Language Models (LLMs) to a given task is a complex challenge. We propose two new unsupervised metrics for faithfulness evaluation using insights from information theory and thermodynamics. Our approach treats an LLM as a bipartite information engine where hidden layers act as a Maxwell demon controlling transformations of context $C $ into answer $A$ via prompt $Q$. We model Question-Context-Answer (QCA) triplets as probability distributions over shared topics. Topic transformations from $C$ to $Q$ and $A$ are modeled as transition matrices ${\bf Q}$ and ${\bf A}$ encoding the query goal and actual result, respectively. Our semantic faithfulness (SF) metric quantifies faithfulness for any given QCA triplet by the Kullback-Leibler (KL) divergence between these matrices. Both matrices are inferred simultaneously via convex optimization of this KL divergence, and the final SF metric is obtained by mapping the minimal divergence onto the unit interval [0,1], where higher scores indicate greater faithfulness. Furthermore, we propose a thermodynamics-based semantic entropy production (SEP) metric in answer generation, and show that high faithfulness generally implies low entropy production. The SF and SEP metrics can be used jointly or separately for LLM evaluation and hallucination control. We demonstrate our framework on LLM summarization of corporate SEC 10-K filings.