WorkForceAgent-R1: Incentivizing Reasoning Capability in LLM-based Web Agents via Reinforcement Learning
This work addresses the problem of generalization and robustness in web agents for enterprise environments, representing an incremental improvement with a novel method for a known bottleneck.
The paper tackles the problem of insufficient reasoning capabilities in LLM-based web agents for enterprise web navigation tasks by introducing WorkForceAgent-R1, which uses a rule-based reinforcement learning framework to enhance reasoning and planning. The result shows that WorkForceAgent-R1 outperforms supervised fine-tuning baselines by 10.26-16.59% on the WorkArena benchmark and achieves competitive performance relative to proprietary LLM-based agents like GPT-4o.
Large language models (LLMs)-empowered web agents enables automating complex, real-time web navigation tasks in enterprise environments. However, existing web agents relying on supervised fine-tuning (SFT) often struggle with generalization and robustness due to insufficient reasoning capabilities when handling the inherently dynamic nature of web interactions. In this study, we introduce WorkForceAgent-R1, an LLM-based web agent trained using a rule-based R1-style reinforcement learning framework designed explicitly to enhance single-step reasoning and planning for business-oriented web navigation tasks. We employ a structured reward function that evaluates both adherence to output formats and correctness of actions, enabling WorkForceAgent-R1 to implicitly learn robust intermediate reasoning without explicit annotations or extensive expert demonstrations. Extensive experiments on the WorkArena benchmark demonstrate that WorkForceAgent-R1 substantially outperforms SFT baselines by 10.26-16.59%, achieving competitive performance relative to proprietary LLM-based agents (gpt-4o) in workplace-oriented web navigation tasks.