LP-LM: No Hallucinations in Question Answering with Logic Programming
This work addresses the problem of hallucinations in LLMs for users who require reliable and accurate question answering.
The authors tackled the problem of hallucinations in large language models (LLMs) by introducing LP-LM, which grounds answers in known facts and produces reliable answers, outperforming current LLMs in accuracy. LP-LM achieves this without hallucinating, even on simple questions.
Large language models (LLMs) are able to generate human-like responses to user queries. However, LLMs exhibit inherent limitations, especially because they hallucinate. This paper introduces LP-LM, a system that grounds answers to questions in known facts contained in a knowledge base (KB), facilitated through semantic parsing in Prolog, and always produces answers that are reliable. LP-LM generates a most probable constituency parse tree along with a corresponding Prolog term for an input question via Prolog definite clause grammar (DCG) parsing. The term is then executed against a KB of natural language sentences also represented as Prolog terms for question answering. By leveraging DCG and tabling, LP-LM runs in linear time in the size of input sentences for sufficiently many grammar rules. Performing experiments comparing LP-LM with current well-known LLMs in accuracy, we show that LLMs hallucinate on even simple questions, unlike LP-LM.