Towards Better & Faster Autoregressive Image Generation: From the Perspective of Entropy
This work addresses the challenge of efficient and high-quality image generation for AI applications, but it is incremental as it builds on existing autoregressive models with novel decoding improvements.
The paper tackled the problem of slow and low-quality autoregressive image generation by identifying that image tokens have lower information density and non-uniform spatial distribution, and it introduced an entropy-informed decoding strategy that improved generation quality and achieved about 85% of the inference cost of conventional methods.
In this work, we first revisit the sampling issues in current autoregressive (AR) image generation models and identify that image tokens, unlike text tokens, exhibit lower information density and non-uniform spatial distribution. Accordingly, we present an entropy-informed decoding strategy that facilitates higher autoregressive generation quality with faster synthesis speed. Specifically, the proposed method introduces two main innovations: 1) dynamic temperature control guided by spatial entropy of token distributions, enhancing the balance between content diversity, alignment accuracy, and structural coherence in both mask-based and scale-wise models, without extra computational overhead, and 2) entropy-aware acceptance rules in speculative decoding, achieving near-lossless generation at about 85\% of the inference cost of conventional acceleration methods. Extensive experiments across multiple benchmarks using diverse AR image generation models demonstrate the effectiveness and generalizability of our approach in enhancing both generation quality and sampling speed.