MegaFusion: Extend Diffusion Models towards Higher-resolution Image Generation without Further Tuning
This addresses a key limitation in text-to-image generation for applications requiring high-resolution outputs, though it is incremental as it builds on existing diffusion models.
The paper tackles the problem of high-resolution image generation in diffusion models, which often suffer from semantic deviations and object replication, by introducing MegaFusion, a method that extends existing models to generate megapixel images with about 40% of the original computational cost without additional fine-tuning.
Diffusion models have emerged as frontrunners in text-to-image generation, but their fixed image resolution during training often leads to challenges in high-resolution image generation, such as semantic deviations and object replication. This paper introduces MegaFusion, a novel approach that extends existing diffusion-based text-to-image models towards efficient higher-resolution generation without additional fine-tuning or adaptation. Specifically, we employ an innovative truncate and relay strategy to bridge the denoising processes across different resolutions, allowing for high-resolution image generation in a coarse-to-fine manner. Moreover, by integrating dilated convolutions and noise re-scheduling, we further adapt the model's priors for higher resolution. The versatility and efficacy of MegaFusion make it universally applicable to both latent-space and pixel-space diffusion models, along with other derivative models. Extensive experiments confirm that MegaFusion significantly boosts the capability of existing models to produce images of megapixels and various aspect ratios, while only requiring about 40% of the original computational cost.