Mashup Learning: Faster Finetuning by Remixing Past Checkpoints
This addresses the problem of computational inefficiency in LLM adaptation for researchers and practitioners, though it is incremental as it builds on existing checkpoint reuse methods.
The paper tackles the problem of inefficient finetuning of LLMs on new tasks by reusing past checkpoints, resulting in improved accuracy by 0.5-5 percentage points and faster convergence with 41-46% fewer training steps.
Finetuning on domain-specific data is a well-established method for enhancing LLM performance on downstream tasks. Training on each dataset produces a new set of model weights, resulting in a multitude of checkpoints saved in-house or on open-source platforms. However, these training artifacts are rarely reused for subsequent experiments despite containing improved model abilities for potentially similar tasks. In this paper, we propose Mashup Learning, a simple method to leverage the outputs of prior training runs to enhance model adaptation to new tasks. Our procedure identifies the most relevant historical checkpoints for a target dataset, aggregates them with model merging, and uses the result as an improved initialization for training. Across 8 standard LLM benchmarks, four models, and two collections of source checkpoints, Mashup Learning consistently improves average downstream accuracy by 0.5-5 percentage points over training from scratch. It also accelerates convergence, requiring 41-46% fewer training steps and up to 37% less total wall-clock time to match from-scratch accuracy, including all selection and merging overhead.