From Sampled Outcomes to Capability Distributions: Rethinking Supervision for LLM Routing
For practitioners deploying multiple LLMs, this work addresses the problem of unreliable routing due to noisy single-response supervision, offering a more robust supervision approach.
Existing LLM routing methods use single responses as supervision, which introduces noise due to stochastic generation. The proposed DARS framework constructs distribution-aware supervision by considering input and output uncertainty, leading to more stable labels and improved routing behavior across diverse tasks.
Existing LLM routing methods typically treat a model's single response to a query as its capability label for training routers. However, because LLM generation is inherently stochastic, such single-shot supervision provides only a noisy observation of a query-model pair's behavior rather than a reliable capability estimate. We show that this assumption introduces systematic noise into routing supervision, making learned routing policies less reliable. To address this issue, we propose DARS (Distribution-Aware Routing Supervision), a framework that constructs routing supervision from a distributional view of model behavior. Instead of relying on a single generated response, DARS considers uncertainty from both the input side and the output side, capturing how semantically equivalent query formulations and stochastic generations affect model performance. Based on these distribution-aware observations, DARS builds more reliable supervision signals for routing. Experiments across diverse tasks show that single-shot labels can be misleading for model selection, while distribution-aware supervision provides more stable labels and improves learned routing behavior. Our results suggest that reliable LLM routing should move beyond single-response observations and be grounded in query-level model capability distributions.