The Unreasonable Effectiveness of Scaling Agents for Computer Use
This work addresses the unreliability of computer-use agents for automating complex digital tasks, representing a strong specific gain rather than a foundational breakthrough.
The paper tackles the problem of unreliable computer-use agents for long-horizon digital tasks by introducing Behavior Best-of-N (bBoN), a scaling method that generates multiple agent rollouts and selects among them using behavior narratives. This approach achieves a state-of-the-art success rate of 69.9% on OSWorld, approaching human-level performance at 72%, with strong generalization to other operating systems.
Computer-use agents (CUAs) hold promise for automating everyday digital tasks, but their unreliability and high variance hinder their application to long-horizon, complex tasks. We introduce Behavior Best-of-N (bBoN), a method that scales over agents by generating multiple rollouts and selecting among them using behavior narratives that describe the agents' rollouts. It enables both wide exploration and principled trajectory selection, substantially improving robustness and success rates. On OSWorld, our bBoN scaling method establishes a new state of the art (SoTA) at 69.9%, significantly outperforming prior methods and approaching human-level performance at 72%, with comprehensive ablations validating key design choices. We further demonstrate strong generalization results to different operating systems on WindowsAgentArena and AndroidWorld. Crucially, our results highlight the unreasonable effectiveness of scaling CUAs, when you do it right: effective scaling requires structured trajectory understanding and selection, and bBoN provides a practical framework to achieve this.