Real-Time Progress Prediction in Reasoning Language Models
This addresses the need for better user oversight and expectation management in complex AI reasoning tasks, though it is incremental as it builds on existing reasoning models.
The paper tackled the problem of opaque internal progress in reasoning language models during extended tasks by developing a method for real-time progress prediction, achieving an average error of 10% for sequences under 16,000 tokens.
Recent advances in reasoning language models -- particularly those that use long, latent chains of thought -- have demonstrated remarkable capabilities in complex, agentic tasks. However, as these models operate over increasingly extended time horizons, their internal progress becomes opaque to users, complicating expectation management and real-time oversight. In this work, we investigate whether real-time progress prediction is feasible. We discretize progress and train a linear probe to classify reasoning states. We then introduce a two-stage fine-tuning approach that enables reasoning models to generate progress estimates (0$\rightarrow$100\%) during inference. Our best fine-tuned model achieves an average error of 10\% for sequences less than 16,000 tokens, offering a practical mechanism for monitoring and interpreting model reasoning in real time.