Hallucination Detection in LLMs: Fast and Memory-Efficient Fine-Tuned Models
This addresses the challenge of deploying reliable LLMs in critical domains like autonomous cars and medicine, though it is incremental as it builds on existing ensemble techniques.
The paper tackles the problem of detecting hallucinations in Large Language Models (LLMs) for high-risk applications by developing a fast and memory-efficient fine-tuning method for ensembles, enabling training and inference on a single GPU.
Uncertainty estimation is a necessary component when implementing AI in high-risk settings, such as autonomous cars, medicine, or insurances. Large Language Models (LLMs) have seen a surge in popularity in recent years, but they are subject to hallucinations, which may cause serious harm in high-risk settings. Despite their success, LLMs are expensive to train and run: they need a large amount of computations and memory, preventing the use of ensembling methods in practice. In this work, we present a novel method that allows for fast and memory-friendly training of LLM ensembles. We show that the resulting ensembles can detect hallucinations and are a viable approach in practice as only one GPU is needed for training and inference.