End-to-End Autoregressive Image Generation with 1D Semantic Tokenizer
This work improves autoregressive image generation by addressing the suboptimal tokenizer issue in two-stage approaches, benefiting the computer vision community.
The paper introduces an end-to-end training pipeline for autoregressive image generation that jointly optimizes a 1D semantic tokenizer and the generative model, achieving a state-of-the-art FID of 1.48 on ImageNet 256x256 without guidance.
Autoregressive image modeling relies on visual tokenizers to compress images into compact latent representations. We design an end-to-end training pipeline that jointly optimizes reconstruction and generation, enabling direct supervision from generation results to the tokenizer. This contrasts with prior two-stage approaches that train tokenizers and generative models separately. We further investigate leveraging vision foundation models to improve 1D tokenizers for autoregressive modeling. Our autoregressive generative model achieves strong empirical results, including a state-of-the-art FID score of 1.48 without guidance on ImageNet 256x256 generation.