Sparse Neurons Carry Strong Signals of Question Ambiguity in LLMs
This addresses the issue of LLMs handling ambiguous queries for users, though it is incremental as it builds on existing neuron analysis methods.
The study tackled the problem of LLMs confidently answering ambiguous questions by showing that question ambiguity is linearly encoded in a few internal neurons, enabling detection and control that outperforms baselines and allows behavior manipulation from answering to abstention.
Ambiguity is pervasive in real-world questions, yet large language models (LLMs) often respond with confident answers rather than seeking clarification. In this work, we show that question ambiguity is linearly encoded in the internal representations of LLMs and can be both detected and controlled at the neuron level. During the model's pre-filling stage, we identify that a small number of neurons, as few as one, encode question ambiguity information. Probes trained on these Ambiguity-Encoding Neurons (AENs) achieve strong performance on ambiguity detection and generalize across datasets, outperforming prompting-based and representation-based baselines. Layerwise analysis reveals that AENs emerge from shallow layers, suggesting early encoding of ambiguity signals in the model's processing pipeline. Finally, we show that through manipulating AENs, we can control LLM's behavior from direct answering to abstention. Our findings reveal that LLMs form compact internal representations of question ambiguity, enabling interpretable and controllable behavior.