Merge and Guide: Unifying Model Merging and Guided Decoding for Controllable Multi-Objective Generation
This addresses the problem of adapting to diverse user needs in generation tasks, offering a more efficient and effective solution than prior merging or decoding-based approaches, though it appears incremental as it builds on existing techniques.
The paper tackles the challenge of controllable multi-objective generation by introducing Merge-And-GuidE (MAGE), a two-stage framework that unifies model merging and guided decoding, resulting in superior controllability, Pareto-optimal performance, and enhanced adaptability compared to existing methods.
Adapting to diverse user needs at test time is a key challenge in controllable multi-objective generation. Existing methods are insufficient: merging-based approaches provide indirect, suboptimal control at the parameter level, often disregarding the impacts of multiple objectives. While decoding-based guidance is more direct, it typically requires aggregating logits from multiple expert models, incurring significant space overhead and relying heavily on individual model capacity. To address these issues, we introduce Merge-And-GuidE (MAGE), a two-stage framework that leverages model merging for guided decoding. We first identify a critical compatibility problem between the guidance and base models. In Stage 1, MAGE resolves this by dynamically constructing a more robust base model, merging a series of backbone models that account for multiple objectives. In Stage 2, we merge explicit and implicit value models into a unified guidance proxy, which then steers the decoding of the base model from Stage 1. Our analysis empirically validates Linear Mode Connectivity (LMC) in value models, explores the relationship between model merging and prediction ensembling, and demonstrates the enhanced controllability afforded by our approach. Extensive experiments show that our method outperforms existing approaches, achieving superior controllability, Pareto-optimal performance, and enhanced adaptability.