Harnessing RLHF for Robust Unanswerability Recognition and Trustworthy Response Generation in LLMs
This addresses the problem of trustworthy response generation in LLMs for conversational information retrieval, though it appears incremental as it builds on existing RLHF and multi-task learning techniques.
The paper tackles the problem of unreliable handling of unanswerable questions in conversational information retrieval systems, which can lead to misleading or hallucinated content from LLMs. The result is SALU, a method that integrates unanswerability detection directly into the LLM's generative process, achieving high accuracy and a dramatic reduction in hallucination through human evaluation.
Conversational Information Retrieval (CIR) systems, while offering intuitive access to information, face a significant challenge: reliably handling unanswerable questions to prevent the generation of misleading or hallucinated content. Traditional approaches often rely on external classifiers, which can introduce inconsistencies with the core generative Large Language Models (LLMs). This paper introduces Self-Aware LLM for Unanswerability (SALU), a novel approach that deeply integrates unanswerability detection directly within the LLM's generative process. SALU is trained using a multi-task learning framework for both standard Question Answering (QA) and explicit abstention generation for unanswerable queries. Crucially, it incorporates a confidence-score-guided reinforcement learning with human feedback (RLHF) phase, which explicitly penalizes hallucinated responses and rewards appropriate abstentions, fostering intrinsic self-awareness of knowledge boundaries. Through extensive experiments on our custom-built C-IR_Answerability dataset, SALU consistently outperforms strong baselines, including hybrid LLM-classifier systems, in overall accuracy for correctly answering or abstaining from questions. Human evaluation further confirms SALU's superior reliability, achieving high scores in factuality, appropriate abstention, and, most importantly, a dramatic reduction in hallucination, demonstrating its ability to robustly "know when to say 'I don't know'."