Detecting (Un)answerability in Large Language Models with Linear Directions
This addresses the issue of unreliable LLM responses for users in QA tasks, though it is incremental as it builds on existing activation-based techniques.
The paper tackles the problem of LLMs hallucinating answers to unanswerable questions by proposing a method to detect unanswerability in extractive QA, showing it generalizes better across datasets than existing approaches and extends to other unanswerability factors.
Large language models (LLMs) often respond confidently to questions even when they lack the necessary information, leading to hallucinated answers. In this work, we study the problem of (un)answerability detection, focusing on extractive question answering (QA) where the model should determine if a passage contains sufficient information to answer a given question. We propose a simple approach for identifying a direction in the model's activation space that captures unanswerability and uses it for classification. This direction is selected by applying activation additions during inference and measuring their impact on the model's abstention behavior. We show that projecting hidden activations onto this direction yields a reliable score for (un)answerability classification. Experiments on two open-weight LLMs and four extractive QA benchmarks show that our method effectively detects unanswerable questions and generalizes better across datasets than existing prompt-based and classifier-based approaches. Moreover, the obtained directions extend beyond extractive QA to unanswerability that stems from factors, such as lack of scientific consensus and subjectivity. Last, causal interventions show that adding or ablating the directions effectively controls the abstention behavior of the model.