VisualPRM: An Effective Process Reward Model for Multimodal Reasoning
This work addresses the challenge of enhancing multimodal reasoning for AI systems, though it appears incremental as it builds on existing PRM and BoN evaluation strategies.
The paper tackles the problem of improving reasoning abilities in Multimodal Large Language Models (MLLMs) by introducing VisualPRM, an 8B-parameter Process Reward Model, which achieves a 5.9-point improvement on benchmarks when applied to a high-capability model like InternVL2.5-78B.
We introduce VisualPRM, an advanced multimodal Process Reward Model (PRM) with 8B parameters, which improves the reasoning abilities of existing Multimodal Large Language Models (MLLMs) across different model scales and families with Best-of-N (BoN) evaluation strategies. Specifically, our model improves the reasoning performance of three types of MLLMs and four different model scales. Even when applied to the highly capable InternVL2.5-78B, it achieves a 5.9-point improvement across seven multimodal reasoning benchmarks. Experimental results show that our model exhibits superior performance compared to Outcome Reward Models and Self-Consistency during BoN evaluation. To facilitate the training of multimodal PRMs, we construct a multimodal process supervision dataset VisualPRM400K using an automated data pipeline. For the evaluation of multimodal PRMs, we propose VisualProcessBench, a benchmark with human-annotated step-wise correctness labels, to measure the abilities of PRMs to detect erroneous steps in multimodal reasoning tasks. We hope that our work can inspire more future research and contribute to the development of MLLMs. Our model, data, and benchmark are released in https://internvl.github.io/blog/2025-03-13-VisualPRM/.