Lavida-O: Elastic Large Masked Diffusion Models for Unified Multimodal Understanding and Generation
This addresses the need for scalable and efficient multimodal AI systems that can handle diverse tasks like object grounding and image editing, representing a new paradigm rather than an incremental improvement.
The paper tackles the problem of limited multimodal understanding and generation in existing models by proposing Lavida-O, a unified Masked Diffusion Model that achieves state-of-the-art performance on benchmarks like RefCOCO, GenEval, and ImgEdit, with high-resolution (1024px) text-to-image synthesis and significant speedup at inference.
We propose Lavida-O, a unified Masked Diffusion Model (MDM) for multimodal understanding and generation. Unlike existing multimodal MDMs such as MMaDa and Muddit which only support simple image-level understanding tasks and low-resolution image generation, Lavida-O presents a single framework that enables image-level understanding, object grounding, image editing, and high-resolution (1024px) text-to-image synthesis. Lavida-O incorporates a novel Elastic Mixture-of-Transformers (Elastic-MoT) architecture that couples a lightweight generation branch with a larger understanding branch, supported by token compression, universal text conditioning and stratified sampling for efficient and high-quality generation. Lavida-O further incorporates planning and iterative self-reflection in image generation and editing tasks, seamlessly boosting generation quality with its understanding capabilities. Lavida-O achieves state-of-the-art performance on a wide range of benchmarks including RefCOCO object grounding, GenEval text-to-image generation, and ImgEdit image editing, outperforming existing autoregressive models and continuous diffusion models such as Qwen2.5-VL and FluxKontext-dev, while offering considerable speedup at inference. These advances establish Lavida-O as a new paradigm for scalable multimodal reasoning and generation.