Ming-Flash-Omni: A Sparse, Unified Architecture for Multimodal Perception and Generation
This work addresses multimodal AI integration for applications requiring unified vision, speech, and language processing, representing a significant but incremental advancement over previous models.
The researchers tackled multimodal perception and generation by proposing Ming-Flash-Omni, a sparse Mixture-of-Experts architecture with 100B total parameters (6.1B active per token), which achieved state-of-the-art results in text-to-image generation and generative segmentation, and set new records on all 12 contextual ASR benchmarks.
We propose Ming-Flash-Omni, an upgraded version of Ming-Omni, built upon a sparser Mixture-of-Experts (MoE) variant of Ling-Flash-2.0 with 100 billion total parameters, of which only 6.1 billion are active per token. This architecture enables highly efficient scaling (dramatically improving computational efficiency while significantly expanding model capacity) and empowers stronger unified multimodal intelligence across vision, speech, and language, representing a key step toward Artificial General Intelligence (AGI). Compared to its predecessor, the upgraded version exhibits substantial improvements across multimodal understanding and generation. We significantly advance speech recognition capabilities, achieving state-of-the-art performance in contextual ASR and highly competitive results in dialect-aware ASR. In image generation, Ming-Flash-Omni introduces high-fidelity text rendering and demonstrates marked gains in scene consistency and identity preservation during image editing. Furthermore, Ming-Flash-Omni introduces generative segmentation, a capability that not only achieves strong standalone segmentation performance but also enhances spatial control in image generation and improves editing consistency. Notably, Ming-Flash-Omni achieves state-of-the-art results in text-to-image generation and generative segmentation, and sets new records on all 12 contextual ASR benchmarks, all within a single unified architecture.