UQLM: A Python Package for Uncertainty Quantification in Large Language Models
This work addresses hallucinations in LLMs to improve safety and trust for downstream applications, but it is incremental as it packages existing UQ methods into a toolkit.
The authors tackled the problem of hallucinations in large language models by introducing UQLM, a Python package that uses uncertainty quantification techniques to detect false or misleading content, providing an off-the-shelf solution with confidence scores from 0 to 1.
Hallucinations, defined as instances where Large Language Models (LLMs) generate false or misleading content, pose a significant challenge that impacts the safety and trust of downstream applications. We introduce UQLM, a Python package for LLM hallucination detection using state-of-the-art uncertainty quantification (UQ) techniques. This toolkit offers a suite of UQ-based scorers that compute response-level confidence scores ranging from 0 to 1. This library provides an off-the-shelf solution for UQ-based hallucination detection that can be easily integrated to enhance the reliability of LLM outputs.