Efficient-VQGAN: Towards High-Resolution Image Generation with Efficient Vision Transformers
This work addresses the problem of high-resolution image generation for computer vision applications, offering an incremental improvement over existing methods by enhancing efficiency and quality.
The paper tackled the challenge of generating high-resolution images efficiently by proposing Efficient-VQGAN, a two-stage framework that uses local attention for quantization and combines global and local attention for feature interaction, resulting in faster generation, higher fidelity, and improved resolution with demonstrated superiority in experiments.
Vector-quantized image modeling has shown great potential in synthesizing high-quality images. However, generating high-resolution images remains a challenging task due to the quadratic computational overhead of the self-attention process. In this study, we seek to explore a more efficient two-stage framework for high-resolution image generation with improvements in the following three aspects. (1) Based on the observation that the first quantization stage has solid local property, we employ a local attention-based quantization model instead of the global attention mechanism used in previous methods, leading to better efficiency and reconstruction quality. (2) We emphasize the importance of multi-grained feature interaction during image generation and introduce an efficient attention mechanism that combines global attention (long-range semantic consistency within the whole image) and local attention (fined-grained details). This approach results in faster generation speed, higher generation fidelity, and improved resolution. (3) We propose a new generation pipeline incorporating autoencoding training and autoregressive generation strategy, demonstrating a better paradigm for image synthesis. Extensive experiments demonstrate the superiority of our approach in high-quality and high-resolution image reconstruction and generation.