PCL-Reasoner-V1.5: Advancing Math Reasoning with Offline Reinforcement Learning
This work addresses the challenge of training stability and efficiency in reinforcement learning for mathematical reasoning in LLMs, representing an incremental advancement.
The authors tackled the problem of improving mathematical reasoning in large language models by proposing an offline reinforcement learning method, achieving state-of-the-art average accuracies of 90.9% on AIME 2024 and 85.6% on AIME 2025.
We present PCL-Reasoner-V1.5, a 32-billion-parameter large language model (LLM) for mathematical reasoning. The model is built upon Qwen2.5-32B and refined via supervised fine-tuning (SFT) followed by reinforcement learning (RL). A central innovation is our proposed offline RL method, which provides superior training stability and efficiency over standard online RL methods such as GRPO. Our model achieves state-of-the-art performance among models post-trained on Qwen2.5-32B, attaining average accuracies of 90.9% on AIME 2024 and 85.6% on AIME 2025. Our work demonstrates offline RL as a stable and efficient paradigm for advancing reasoning in LLMs. All experiments were conducted on Huawei Ascend 910C NPUs.