TagRouter: Learning Route to LLMs through Tags for Open-Domain Text Generation Tasks
This addresses scalability and cost-efficiency challenges in model routing for the LLM community, offering an incremental improvement over existing methods.
The paper tackles the problem of model routing for large language models (LLMs) in open-domain text generation by proposing TagRouter, a training-free method that improves system performance and reduces costs, achieving a 6.15% increase in accept rate and a 17.20% cost reduction compared to baselines.
Model routing allocates queries to the suitable model, improving system performance while reducing costs. However, existing routing methods face practical limitations that hinder scalability in large-scale applications and struggle to keep up with the rapid growth of the large language model (LLM) ecosystem. To tackle these challenges, we propose TagRouter, a training-free model routing method designed to optimize the synergy among multiple LLMs for open-domain text generation tasks. Experimental results demonstrate that TagRouter outperforms 13 baseline methods, increasing the accept rate of system by 6.15% and reducing costs by 17.20%, achieving optimal cost-efficiency. Our findings provides the LLM community with an efficient and scalable solution for model ensembling, offering users an evolvable "super model."