Read Before You Think: Mitigating LLM Comprehension Failures with Step-by-Step Reading
This addresses a core bottleneck in LLM reasoning for tasks requiring deep comprehension, offering an efficient method to improve performance, though it is incremental as it builds on existing prompting techniques.
The paper tackles LLM comprehension failures in complex reasoning by introducing Step-by-Step Reading (SSR++) prompts, which guide models to parse questions more finely and refocus attention, achieving new state-of-the-art results on multiple reasoning benchmarks.
Large Language Models (LLMs) often fail on complex reasoning tasks due to flawed question comprehension, not just flawed logic. This paper presents a systematic investigation into these comprehension failures. Our work yields three key insights: (1) the step-by-step principle, effective for calculation, can be migrated to the reading process to enhance comprehension; (2) increasing the proportion of question-related tokens (e.g., via repetition) succeeds by refocusing attention, a mechanism that can be explicitly controlled; and (3) backward dependencies represent a core bottleneck for decoder-only models that persists even with strong methods like Chain-of-Thought. Based on these findings, we introduce the Step-by-Step Reading (SSR) family of prompts. This multi-stage approach culminates in SSR++, a method specifically engineered to deepen model comprehension by guiding it to parse questions with finer granularity, focus attention on critical tokens, and resolve backward dependencies through iterative re-contextualization. SSR++ sets a new state-of-the-art on multiple reasoning benchmarks, and our analysis confirms it works by directly mitigating semantic misunderstanding. These results demonstrate that guiding how a model reads is a powerful and efficient method for improving its reasoning ability.