ProFuser: Progressive Fusion of Large Language Models
This work addresses the problem of creating more powerful and versatile language models for AI applications, but it is incremental as it builds on existing fusion methods by enhancing the evaluation process.
The paper tackled the challenge of selecting advantageous models during the fusion of large language models by introducing ProFuser, which incorporates both training and inference modes for a more comprehensive assessment, resulting in improved performance in knowledge, reasoning, and safety compared to baseline methods when fusing three models.
While fusing the capacities and advantages of various large language models offers a pathway to construct more powerful and versatile models, a fundamental challenge is to properly select advantageous model during training. Existing fusion methods primarily focus on the training mode that uses cross entropy on ground truth in a teacher-forcing setup to measure a model's advantage, which may provide limited insight towards model advantage. In this paper, we introduce a novel approach that enhances the fusion process by incorporating both the training and inference modes. Our method evaluates model advantage not only through cross entropy during training but also by considering inference outputs, providing a more comprehensive assessment. To combine the two modes effectively, we introduce ProFuser to progressively transition from inference mode to training mode. To validate ProFuser's effectiveness, we fused three models, including Vicuna-7B-v1.5, Llama-2-7B-Chat, and MPT-7B-8K-Chat, and demonstrated the improved performance in knowledge, reasoning, and safety compared to baseline methods.