ECVL-ROUTER: Scenario-Aware Routing for Vision-Language Models
This addresses efficiency and cost issues for users of vision-language models by optimizing model selection based on scenario needs, though it is incremental as it builds on existing routing concepts.
The paper tackles the problem of varying user requirements in vision-language tasks by proposing ECVL-ROUTER, a scenario-aware routing framework that dynamically selects between large and small models, achieving over 80% of queries routed to small models with less than a 10% drop in problem-solving probability.
Vision-Language Models (VLMs) excel in diverse multimodal tasks. However, user requirements vary across scenarios, which can be categorized into fast response, high-quality output, and low energy consumption. Relying solely on large models deployed in the cloud for all queries often leads to high latency and energy cost, while small models deployed on edge devices are capable of handling simpler tasks with low latency and energy cost. To fully leverage the strengths of both large and small models, we propose ECVL-ROUTER, the first scenario-aware routing framework for VLMs. Our approach introduces a new routing strategy and evaluation metrics that dynamically select the appropriate model for each query based on user requirements, maximizing overall utility. We also construct a multimodal response-quality dataset tailored for router training and validate the approach through extensive experiments. Results show that our approach successfully routes over 80\% of queries to the small model while incurring less than 10\% drop in problem solving probability.