Know the Unknown: An Uncertainty-Sensitive Method for LLM Instruction Tuning
This addresses the issue of hallucinations in LLMs for users relying on accurate and reliable AI responses, representing a strong specific gain rather than a foundational breakthrough.
The paper tackles the problem of LLMs struggling to admit knowledge gaps, which leads to hallucinations, by introducing US-Tuning, a two-stage method for instruction tuning that enhances knowledge boundary awareness and instruction adherence. The fine-tuned Llama2-7B model shows a 34.7% improvement in handling out-of-knowledge questions and outperforms GPT-4 by 4.2% in overall performance.
Large language models (LLMs) demonstrate remarkable capabilities but face challenges from hallucinations, which typically arise from insufficient knowledge or context. While instructing LLMs to acknowledge knowledge limitations by responding with "I don't know" appears promising, we find that models consistently struggle with admitting knowledge gaps. This challenge may originate from current instruction datasets that emphasise answer generation over knowledge boundary awareness. To address this limitation, we introduce Uncertainty-and-Sensitivity-Aware Tuning (US-Tuning), a novel two-stage approach for contextual question answering (QA). The first stage enhances LLMs' ability to recognise their knowledge boundaries, while the second stage reinforces instruction adherence through carefully designed causal prompts. Our experimental results demonstrate that US-Tuning not only significantly reduces incorrect answers in contextual QA but also improves models' faithfulness to their parametric knowledge, mitigating hallucinations in general QA tasks. Our fine-tuned Llama2-7B model achieves up to a 34.7% improvement in handling out-of-knowledge questions and outperforms GPT-4 by 4.2% in overall performance.