Adaptive Milestone Reward for GUI Agents
This addresses the challenge of reward design for long-horizon tasks in GUI automation, offering a method that balances fidelity and density, though it appears incremental as it builds on existing RL paradigms.
The paper tackles the temporal credit assignment problem in reinforcement learning for mobile GUI agents by proposing the Adaptive Milestone Reward (ADMIRE) mechanism, which achieves over 10% absolute improvement in success rate on AndroidWorld.
Reinforcement Learning (RL) has emerged as a mainstream paradigm for training Mobile GUI Agents, yet it struggles with the temporal credit assignment problem inherent in long-horizon tasks. A primary challenge lies in the trade-off between reward fidelity and density: outcome reward offers high fidelity but suffers from signal sparsity, while process reward provides dense supervision but remains prone to bias and reward hacking. To resolve this conflict, we propose the Adaptive Milestone Reward (ADMIRE) mechanism. ADMIRE constructs a verifiable, adaptive reward system by anchoring trajectory to milestones, which are dynamically distilled from successful explorations. Crucially, ADMIRE integrates an asymmetric credit assignment strategy that denoises successful trajectories and scaffolds failed trajectories. Extensive experiments demonstrate that ADMIRE consistently yields over 10% absolute improvement in success rate across different base models on AndroidWorld. Moreover, the method exhibits robust generalizability, achieving strong performance across diverse RL algorithms and heterogeneous environments such as web navigation and embodied tasks.