Timely Machine: Awareness of Time Makes Test-Time Scaling Agentic
This work addresses the challenge of adapting large language models to time-constrained reasoning in agentic applications, offering a new perspective on test-time scaling.
The paper tackles the problem of test-time scaling in agentic scenarios where tool latency disrupts traditional generation-length-based approaches, proposing Timely Machine to redefine test-time as wall-clock time and introducing Timely-RL to improve time budget awareness, which consistently boosts performance across a new benchmark.
As large language models (LLMs) increasingly tackle complex reasoning tasks, test-time scaling has become critical for enhancing capabilities. However, in agentic scenarios with frequent tool calls, the traditional generation-length-based definition breaks down: tool latency decouples inference time from generation length. We propose Timely Machine, redefining test-time as wall-clock time, where models dynamically adjust strategies based on time budgets. We introduce Timely-Eval, a benchmark spanning high-frequency tool calls, low-frequency tool calls, and time-constrained reasoning. By varying tool latency, we find smaller models excel with fast feedback through more interactions, while larger models dominate high-latency settings via superior interaction quality. Moreover, existing models fail to adapt reasoning to time budgets. We propose Timely-RL to address this gap. After cold-start supervised fine-tuning, we use reinforcement learning to enhance temporal planning. Timely-RL improves time budget awareness and consistently boosts performance across Timely-Eval. We hope our work offers a new perspective on test-time scaling for the agentic era.