MAP: Low-compute Model Merging with Amortized Pareto Fronts via Quadratic Approximation
This work addresses the need for flexible solutions in real-world applications where practitioners must balance competing task objectives during model merging, though it is incremental as it builds on existing merging methods.
The paper tackles the problem of model merging for multitask models by introducing MAP, a low-compute algorithm that efficiently identifies a Pareto set of scaling coefficients to balance trade-offs between tasks, with experimental results showing accurate Pareto front identification on vision and NLP tasks.
Model merging has emerged as an effective approach to combine multiple single-task models into a multitask model. This process typically involves computing a weighted average of the model parameters without any additional training. Existing model-merging methods focus on enhancing average task accuracy. However, interference and conflicts between the objectives of different tasks can lead to trade-offs during the merging process. In real-world applications, a set of solutions with various trade-offs can be more informative, helping practitioners make decisions based on diverse preferences. In this paper, we introduce a novel and low-compute algorithm, Model Merging with Amortized Pareto Front (MAP). MAP efficiently identifies a Pareto set of scaling coefficients for merging multiple models, reflecting the trade-offs involved. It amortizes the substantial computational cost of evaluations needed to estimate the Pareto front by using quadratic approximation surrogate models derived from a pre-selected set of scaling coefficients. Experimental results on vision and natural language processing tasks demonstrate that MAP can accurately identify the Pareto front, providing practitioners with flexible solutions to balance competing task objectives. We also introduce Bayesian MAP for scenarios with a relatively low number of tasks and Nested MAP for situations with a high number of tasks, further reducing the computational cost of evaluation.