Task-Aware LLM Routing with Multi-Level Task-Profile-Guided Data Synthesis for Cold-Start Scenarios
This work addresses the challenge of selecting cost-effective LLMs for users in scenarios lacking in-domain training data, representing an incremental improvement over existing routing methods.
The paper tackles the problem of poor generalization in LLM routing systems under cold-start scenarios by introducing a multi-level task-profile-guided data synthesis framework and TRouter, a task-type-aware router, which together enhance routing utility and alleviate cold-start issues across multiple benchmarks.
Large language models (LLMs) exhibit substantial variability in performance and computational cost across tasks and queries, motivating routing systems that select models to meet user-specific cost-performance trade-offs. However, existing routers generalize poorly in cold-start scenarios where in-domain training data is unavailable. We address this limitation with a multi-level task-profile-guided data synthesis framework that constructs a hierarchical task taxonomy and produces diverse question-answer pairs to approximate the test-time query distribution. Building on this, we introduce TRouter, a task-type-aware router approach that models query-conditioned cost and performance via latent task-type variables, with prior regularization derived from the synthesized task taxonomy. This design enhances TRouter's routing utility under both cold-start and in-domain settings. Across multiple benchmarks, we show that our synthesis framework alleviates cold-start issues and that TRouter delivers effective LLM routing.