Efficient Uncertainty Estimation for LLM-based Entity Linking in Tabular Data
This addresses the need for reliable uncertainty estimates in real-world entity linking applications, offering a cost-effective solution for data integration and enrichment.
The paper tackles the problem of efficiently estimating uncertainty for LLM-based entity linking in tabular data, showing that a self-supervised method using token-level features reduces computational cost while effectively detecting low-accuracy outputs.
Linking textual values in tabular data to their corresponding entities in a Knowledge Base is a core task across a variety of data integration and enrichment applications. Although Large Language Models (LLMs) have shown State-of-The-Art performance in Entity Linking (EL) tasks, their deployment in real-world scenarios requires not only accurate predictions but also reliable uncertainty estimates, which require resource-demanding multi-shot inference, posing serious limits to their actual applicability. As a more efficient alternative, we investigate a self-supervised approach for estimating uncertainty from single-shot LLM outputs using token-level features, reducing the need for multiple generations. Evaluation is performed on an EL task on tabular data across multiple LLMs, showing that the resulting uncertainty estimates are highly effective in detecting low-accuracy outputs. This is achieved at a fraction of the computational cost, ultimately supporting a cost-effective integration of uncertainty measures into LLM-based EL workflows. The method offers a practical way to incorporate uncertainty estimation into EL workflows with limited computational overhead.