Zero-Shot Multi-task Hallucination Detection
This addresses the need for robust evaluation of text generation quality in AI, though it is incremental as it builds on existing hallucination detection methods.
The paper tackles the problem of hallucination in large language models by proposing a zero-shot detection framework, achieving accuracies of 0.78 in model-aware and 0.61 in model-agnostic settings while being computationally efficient.
In recent studies, the extensive utilization of large language models has underscored the importance of robust evaluation methodologies for assessing text generation quality and relevance to specific tasks. This has revealed a prevalent issue known as hallucination, an emergent condition in the model where generated text lacks faithfulness to the source and deviates from the evaluation criteria. In this study, we formally define hallucination and propose a framework for its quantitative detection in a zero-shot setting, leveraging our definition and the assumption that model outputs entail task and sample specific inputs. In detecting hallucinations, our solution achieves an accuracy of 0.78 in a model-aware setting and 0.61 in a model-agnostic setting. Notably, our solution maintains computational efficiency, requiring far less computational resources than other SOTA approaches, aligning with the trend towards lightweight and compressed models.