DecoPrompt : Decoding Prompts Reduces Hallucinations when Large Language Models Meet False Premises
This addresses a critical issue for users relying on LLMs for factual accuracy, though it is an incremental improvement in mitigating hallucinations.
The paper tackles the problem of large language models (LLMs) generating hallucinated outputs when given false premises, by proposing DecoPrompt, a prompting algorithm that reduces hallucinations effectively, as demonstrated in experiments on two datasets with different LLMs.
While large language models (LLMs) have demonstrated increasing power, they have also called upon studies on their hallucinated outputs that deviate from factually correct statements. In this paper, we focus on one important scenario of false premises, where LLMs are distracted by misaligned claims although the model possesses the required factual knowledge to answer original questions accurately. Inspired by the observation that entropy of the false-premise prompt is closely related to its likelihood to elicit hallucination generation, we propose a new prompting algorithm, named DecoPrompt, to mitigate hallucination. DecoPrompt leverages LLMs to "decode" the false-premise prompts without really eliciting hallucination output from LLMs. We perform experiments on two datasets, demonstrating that DecoPrompt can reduce hallucinations effectively on outputs from different LLMs. Moreover, DecoPrompt exhibits cross-model transferability, which facilitates its applications to scenarios such as LLMs of large sizes or unavailable model logits.