xRouter: Training Cost-Aware LLMs Orchestration System via Reinforcement Learning
This work addresses cost inefficiencies in LLM orchestration for developers and organizations, offering a practical solution that is incremental by building on existing routing and reinforcement learning techniques.
The paper tackles the problem of high costs in LLM deployments by introducing xRouter, a reinforcement learning-based system that dynamically routes tasks between expensive and lightweight models to optimize cost-performance trade-offs, achieving substantial cost reductions while maintaining comparable task completion rates.
Modern LLM deployments confront a widening cost-performance spectrum: premium models deliver strong reasoning but are expensive, while lightweight models are economical yet brittle on complex tasks. Static escalation rules and keyword heuristics under-utilize this spectrum and fail to adapt across task types. We present xRouter, a tool-calling-based routing system in which a learned router can either answer directly or invoke one or more external models. The router is trained end-to-end with reinforcement learning using an explicit, cost-aware reward that encodes cost-performance trade-offs, eliminating the need for hand-engineered routing rules. Our implementation encompasses the full reinforcement learning framework, including reward and cost accounting, as well as the deployment and evaluation pipelines. Across diverse benchmarks, xRouter achieves strong cost-performance trade-offs (e.g., substantial cost reductions at comparable task completion rates), and provides empirical insights into what reliably helps learned routing and what does not, ranging from model trainability to the difficulty of eliciting sophisticated orchestration behaviors in small open models. We hope these findings and our open implementation will serve as a practical substrate for advancing learned, cost-aware LLM orchestration.