SuperMerge: An Approach For Gradient-Based Model Merging
This addresses the incremental need for solving newer tasks in high-throughput applications with lower latency and cost, though it is an incremental improvement over existing model merging methods.
The paper tackles the problem of efficiently updating deployed task-specific models for new tasks without expensive retraining, proposing SuperMerge, a gradient-based model merging method that achieves similar performance to fully fine-tuned models on all tasks while being lightweight and fast.
Large language models, such as ChatGPT, Claude, or LLaMA, are gigantic, monolithic, and possess the superpower to simultaneously support thousands of tasks. However, high-throughput applications often prefer smaller task-specific models because of their lower latency and cost. One challenge of using task-specific models is the incremental need for solving newer tasks after the model is already deployed for existing tasks. A straightforward solution requires fine-tuning the model again for both existing and new tasks, which is computationally expensive and time-consuming. To address this issue, we propose a model merging based approach called SUPERMERGE. SUPERMERGE is a gradient-based method to systematically merge several fine-tuned models trained on existing and new tasks. SUPERMERGE is designed to be lightweight and fast, and the merged model achieves similar performance to fully fine-tuned models on all tasks. Furthermore, we proposed a hierarchical model merging strategy to reduce the peak space requirement without sacrificing the performance of the merged model. We experimentally demonstrate that SUPERMERGE outperforms existing model merging methods on common natural language processing and computer vision tasks.