Direction-Aware Diagonal Autoregressive Image Generation
This addresses a specific bottleneck in autoregressive image generation for computer vision researchers, offering a novel scanning order and direction-aware modules to improve performance.
The paper tackled the problem of raster-ordered image token sequences having large Euclidean distances between adjacent tokens at line breaks, which hinders autoregressive generation, by proposing a Direction-Aware Diagonal Autoregressive method that generates tokens in a diagonal scanning order; their largest model (2.0B parameters) achieved a state-of-the-art FID score of 1.37 on the 256×256 ImageNet benchmark.
The raster-ordered image token sequence exhibits a significant Euclidean distance between index-adjacent tokens at line breaks, making it unsuitable for autoregressive generation. To address this issue, this paper proposes Direction-Aware Diagonal Autoregressive Image Generation (DAR) method, which generates image tokens following a diagonal scanning order. The proposed diagonal scanning order ensures that tokens with adjacent indices remain in close proximity while enabling causal attention to gather information from a broader range of directions. Additionally, two direction-aware modules: 4D-RoPE and direction embeddings are introduced, enhancing the model's capability to handle frequent changes in generation direction. To leverage the representational capacity of the image tokenizer, we use its codebook as the image token embeddings. We propose models of varying scales, ranging from 485M to 2.0B. On the 256$\times$256 ImageNet benchmark, our DAR-XL (2.0B) outperforms all previous autoregressive image generators, achieving a state-of-the-art FID score of 1.37.