The LLM Already Knows: Estimating LLM-Perceived Question Difficulty via Hidden Representations
This addresses the need for efficient and accurate difficulty estimation in LLMs to enhance performance evaluation and adaptive inference, though it is incremental as it builds on existing methods.
The paper tackled the problem of estimating question difficulty as perceived by large language models (LLMs) by proposing a method that uses only hidden representations, achieving higher accuracy and efficiency than existing baselines across textual and multimodal tasks, and applied it to improve inference strategies like Self-Consistency, reducing generated tokens.
Estimating the difficulty of input questions as perceived by large language models (LLMs) is essential for accurate performance evaluation and adaptive inference. Existing methods typically rely on repeated response sampling, auxiliary models, or fine-tuning the target model itself, which may incur substantial computational costs or compromise generality. In this paper, we propose a novel approach for difficulty estimation that leverages only the hidden representations produced by the target LLM. We model the token-level generation process as a Markov chain and define a value function to estimate the expected output quality given any hidden state. This allows for efficient and accurate difficulty estimation based solely on the initial hidden state, without generating any output tokens. Extensive experiments across both textual and multimodal tasks demonstrate that our method consistently outperforms existing baselines in difficulty estimation. Moreover, we apply our difficulty estimates to guide adaptive reasoning strategies, including Self-Consistency, Best-of-N, and Self-Refine, achieving higher inference efficiency with fewer generated tokens.