ProRe: A Proactive Reward System for GUI Agents via Reasoner-Actor Collaboration
This addresses the challenge of evaluating and training GUI agents where traditional reward methods fail, offering a domain-specific solution with measurable gains.
The paper tackled the problem of inaccurate reward systems for GUI agents by proposing ProRe, a proactive reward system using reasoner-actor collaboration, which improved reward accuracy by up to 5.3% and success rates by up to 22.4% in experiments.
Reward is critical to the evaluation and training of large language models (LLMs). However, existing rule-based or model-based reward methods struggle to generalize to GUI agents, where access to ground-truth trajectories or application databases is often unavailable, and static trajectory-based LLM-as-a-Judge approaches suffer from limited accuracy. To address these challenges, we propose ProRe, a proactive reward system that leverages a general-purpose reasoner and domain-specific evaluator agents (actors). The reasoner schedules targeted state probing tasks, which the evaluator agents then execute by actively interacting with the environment to collect additional observations. This enables the reasoner to assign more accurate and verifiable rewards to GUI agents. Empirical results on over 3K trajectories demonstrate that ProRe improves reward accuracy and F1 score by up to 5.3% and 19.4%, respectively. Furthermore, integrating ProRe with state-of-the-art policy agents yields a success rate improvement of up to 22.4%.