SpectR: Dynamically Composing LM Experts with Spectral Routing
This addresses the problem of efficiently utilizing existing expert models for AI practitioners, though it is incremental as it builds on prior work in model composition and routing.
The paper tackles the challenge of effectively leveraging specialized expert language models for real-world tasks by introducing SPECTR, a method for dynamically composing these experts during inference without additional training, which improves routing accuracy and task performance across domains.
Training large, general-purpose language models poses significant challenges. The growing availability of specialized expert models, fine-tuned from pretrained models for specific tasks or domains, offers a promising alternative. Leveraging the potential of these existing expert models in real-world applications requires effective methods to select or merge the models best suited for a given task. This paper introduces SPECTR, an approach for dynamically composing expert models at each time step during inference. Notably, our method requires no additional training and enables flexible, token- and layer-wise model combinations. Our experimental results demonstrate that SPECTR improves routing accuracy over alternative training-free methods, increasing task performance across expert domains.