UniGen: Enhanced Training & Test-Time Strategies for Unified Multimodal Understanding and Generation
This work addresses the challenge of creating efficient and high-quality unified MLLMs for multimodal AI applications, representing an incremental advancement with specific improvements in test-time scaling.
The paper tackles the problem of building a unified multimodal large language model (MLLM) for both image understanding and generation by proposing enhanced training and test-time strategies, achieving state-of-the-art performance with scores of 0.78 on GenEval and 85.19 on DPG-Bench.
We introduce UniGen, a unified multimodal large language model (MLLM) capable of image understanding and generation. We study the full training pipeline of UniGen from a data-centric perspective, including multi-stage pre-training, supervised fine-tuning, and direct preference optimization. More importantly, we propose a new Chain-of-Thought Verification (CoT-V) strategy for test-time scaling, which significantly boosts UniGen's image generation quality using a simple Best-of-N test-time strategy. Specifically, CoT-V enables UniGen to act as both image generator and verifier at test time, assessing the semantic alignment between a text prompt and its generated image in a step-by-step CoT manner. Trained entirely on open-source datasets across all stages, UniGen achieves state-of-the-art performance on a range of image understanding and generation benchmarks, with a final score of 0.78 on GenEval and 85.19 on DPG-Bench. Through extensive ablation studies, our work provides actionable insights and addresses key challenges in the full life cycle of building unified MLLMs, contributing meaningful directions to the future research.