TTSR: Test-Time Self-Reflection for Continual Reasoning Improvement
This addresses the challenge of adapting LLMs to specific reasoning weaknesses during test time, offering a stable method for continual improvement, though it appears incremental as it builds on existing test-time training paradigms.
The paper tackled the problem of unreliable pseudo-labels and inefficient learning in test-time training for large language models by proposing TTSR, a self-reflective framework that alternates between student and teacher roles, resulting in consistent improvements in reasoning performance across multiple benchmarks.
Test-time Training enables model adaptation using only test questions and offers a promising paradigm for improving the reasoning ability of large language models (LLMs). However, it faces two major challenges: test questions are often highly difficult, making self-generated pseudo-labels unreliable, and existing methods lack effective mechanisms to adapt to a model's specific reasoning weaknesses, leading to inefficient learning. To address these issues, we propose \textbf{TTSR}, a self-reflective test-time self-evolving training framework. TTSR employs a single pretrained language model that alternates between the roles of a \textit{Student} and a \textit{Teacher} at test time. The Student focuses on solving problems and learning from synthesized variant questions, while the Teacher analyzes the Student's failed reasoning trajectories, summarizes recurring reasoning weaknesses, and synthesizes targeted variant questions accordingly. This process guides the model to improve within a learnable regime through a continual self-evolving loop. Experimental results on multiple challenging mathematical reasoning benchmarks show that TTSR consistently improves reasoning performance and generalizes well across different model backbones and general-domain reasoning tasks. These findings suggest that teacher-mediated self-reflection provides an effective pathway for stable and continual reasoning improvement at test time.