Stepwise Think-Critique: A Unified Framework for Robust and Interpretable LLM Reasoning
This addresses the need for more robust and interpretable reasoning in LLMs, though it appears incremental as it builds on existing reinforcement learning and self-critique methods.
The paper tackled the problem of large language models (LLMs) decoupling reasoning from verification, which lacks immediate feedback or increases complexity, by proposing Stepwise Think-Critique (STC), a unified framework that interleaves reasoning and self-critique at each step; experiments on mathematical reasoning benchmarks showed it produces more interpretable reasoning traces.
Human beings solve complex problems through critical thinking, where reasoning and evaluation are intertwined to converge toward correct solutions. However, most existing large language models (LLMs) decouple reasoning from verification: they either generate reasoning without explicit self-checking or rely on external verifiers to detect errors post hoc. The former lacks immediate feedback, while the latter increases system complexity and hinders synchronized learning. Motivated by human critical thinking, we propose Stepwise Think-Critique (STC), a unified framework that interleaves reasoning and self-critique at each step within a single model. STC is trained with a hybrid reinforcement learning objective combining reasoning rewards and critique-consistency rewards to jointly optimize reasoning quality and self-evaluation. Experiments on mathematical reasoning benchmarks show that STC demonstrates strong critic-thinking capabilities and produces more interpretable reasoning traces, representing a step toward LLMs with built-in critical thinking.