LoopCoder-v2: Only Loop Once for Efficient Test-Time Computation Scaling
This paper provides a practical analysis and diagnostic for selecting loop count in parallel loop Transformers, addressing the trade-off between refinement gains and positional mismatch costs for code generation tasks.
LoopCoder-v2 introduces Parallel Loop Transformers (PLT) with shared-KV gated sliding-window attention to reduce latency and memory costs of looped Transformers. Training a 7B model with two loops improves SWE-bench Verified from 43.0 to 64.4 and Multi-SWE from 14.0 to 31.0, but three or more loops cause regression due to a gain-cost trade-off.
Looped Transformers scale latent computation by repeatedly applying shared blocks, but sequential looping increases latency and KV-cache memory with the loop count. Parallel loop Transformers (PLT) alleviate this cost through cross-loop position offsets (CLP) and shared-KV gated sliding-window attention, making loop count a practical design choice. We therefore study PLT loop-count selection through a gain--cost view: an extra loop may refine representations, but CLP also introduces a positional mismatch at each loop boundary. We instantiate this study by training LoopCoder-v2, a family of 7B PLT coders with different loop counts, from scratch on 18T tokens, followed by matched instruction tuning and evaluation. Empirically, the two-loop variant delivers broad gains over the non-looped baseline across code generation, code reasoning, agentic software engineering, and tool-use benchmarks, improving SWE-bench Verified from 43.0 to 64.4 points and Multi-SWE from 14.0 to 31.0 points. In contrast, variants with three or more loops regress, revealing a strongly non-monotonic loop-count effect. Our diagnostics show that loop 2 provides the main productive refinement, while later loops yield diminishing, oscillatory updates and reduced representational diversity. Because the CLP-induced mismatch remains roughly fixed as refinement gains shrink, the offset cost increasingly dominates. This gain--cost trade-off explains PLT's saturation at two loops and provides diagnostics for loop-count selection.