Resurrect Mask AutoRegressive Modeling for Efficient and Scalable Image Generation
This work addresses efficiency and scalability issues in image generation for AI researchers and practitioners, offering a significant speed-up with competitive quality.
The study tackled the underperformance of Masked AutoRegressive (MAR) models in image generation by refining the architecture into MaskGIL, achieving a FID score of 3.71 on ImageNet 256x256, matching state-of-the-art AR models while reducing inference steps from 256 to 8.
AutoRegressive (AR) models have made notable progress in image generation, with Masked AutoRegressive (MAR) models gaining attention for their efficient parallel decoding. However, MAR models have traditionally underperformed when compared to standard AR models. This study refines the MAR architecture to improve image generation quality. We begin by evaluating various image tokenizers to identify the most effective one. Subsequently, we introduce an improved Bidirectional LLaMA architecture by replacing causal attention with bidirectional attention and incorporating 2D RoPE, which together form our advanced model, MaskGIL. Scaled from 111M to 1.4B parameters, MaskGIL achieves a FID score of 3.71, matching state-of-the-art AR models in the ImageNet 256x256 benchmark, while requiring only 8 inference steps compared to the 256 steps of AR models. Furthermore, we develop a text-driven MaskGIL model with 775M parameters for generating images from text at various resolutions. Beyond image generation, MaskGIL extends to accelerate AR-based generation and enable real-time speech-to-image conversion. Our codes and models are available at https://github.com/synbol/MaskGIL.