LGNov 1, 2024

MoD: A Distribution-Based Approach for Merging Large Language Models

arXiv:2411.00406v1h-index: 2Has Code
Originality Incremental advance
AI Analysis

This work addresses resource and operational efficiency problems for developers and users of multiple specialized LLMs, though it appears incremental as it builds on existing model merging techniques.

The paper tackles the challenge of efficiently merging specialized large language models (LLMs) by proposing the Mixture of Distributions (MoD) framework, which operates on output probability distributions instead of model weights, and demonstrates that it significantly outperforms existing merging techniques on mathematical reasoning benchmarks using Qwen2.5 models.

Large language models (LLMs) have enabled the development of numerous specialized, task-specific variants. However, the maintenance and deployment of these individual models present substantial challenges in terms of resource utilization and operational efficiency. In this work, we propose the \textit{Mixture of Distributions (MoD)} framework, a novel approach for merging LLMs that operates directly on their output probability distributions, rather than on model weights. Unlike traditional weight-averaging methods, MoD effectively preserves the specialized capabilities of individual models while enabling efficient knowledge sharing across tasks. Through extensive experimentation on mathematical reasoning benchmarks using Qwen2.5 models, we demonstrate that MoD significantly outperforms existing model merging techniques across multiple benchmarks. All code, data, and experimental materials are published at https://github.com/knovel-eng/mod.

Code Implementations2 repos
Foundations

The foundational work for this paper's niche, ranked by how specifically the neighbourhood builds on it — not by global fame.

Your Notes