Agentic Test-Time Scaling for WebAgents
This addresses efficiency and reliability issues for researchers and practitioners deploying web agents in long-horizon tasks, though it is incremental as it builds on existing test-time scaling methods.
The paper tackled the problem of inefficient compute allocation in test-time scaling for multi-step agents, where naive uniform scaling shows diminishing returns, and introduced CATTS, a confidence-aware method that dynamically allocates compute based on vote-derived uncertainty, improving performance by up to 9.1% while using up to 2.3x fewer tokens.
Test-time scaling has become a standard way to improve performance and boost reliability of neural network models. However, its behavior on agentic, multi-step tasks remains less well-understood: small per-step errors can compound over long horizons; and we find that naive policies that uniformly increase sampling show diminishing returns. In this work, we present CATTS, a simple technique for dynamically allocating compute for multi-step agents. We first conduct an empirical study of inference-time scaling for web agents. We find that uniformly increasing per-step compute quickly saturates in long-horizon environments. We then investigate stronger aggregation strategies, including an LLM-based Arbiter that can outperform naive voting, but that can overrule high-consensus decisions. We show that uncertainty statistics derived from the agent's own vote distribution (entropy and top-1/top-2 margin) correlate with downstream success and provide a practical signal for dynamic compute allocation. Based on these findings, we introduce Confidence-Aware Test-Time Scaling (CATTS), which uses vote-derived uncertainty to allocate compute only when decisions are genuinely contentious. CATTS improves performance on WebArena-Lite and GoBrowse by up to 9.1% over React while using up to 2.3x fewer tokens than uniform scaling, providing both efficiency gains and an interpretable decision rule.