APT: Improving Diffusion Models for High Resolution Image Generation with Adaptive Path Tracing
This work provides a practical solution for generating high-resolution images without extensive retraining, benefiting applications in digital art and media production, though it is incremental over existing patch-based approaches.
The paper tackles the problem of high-resolution image generation with diffusion models by addressing patch-level distribution shift and increased patch monotonicity in training-free patch-based methods, resulting in clearer details and faster sampling with minimal quality degradation.
Latent Diffusion Models (LDMs) are generally trained at fixed resolutions, limiting their capability when scaling up to high-resolution images. While training-based approaches address this limitation by training on high-resolution datasets, they require large amounts of data and considerable computational resources, making them less practical. Consequently, training-free methods, particularly patch-based approaches, have become a popular alternative. These methods divide an image into patches and fuse the denoising paths of each patch, showing strong performance on high-resolution generation. However, we observe two critical issues for patch-based approaches, which we call ``patch-level distribution shift" and ``increased patch monotonicity." To address these issues, we propose Adaptive Path Tracing (APT), a framework that combines Statistical Matching to ensure patch distributions remain consistent in upsampled latents and Scale-aware Scheduling to deal with the patch monotonicity. As a result, APT produces clearer and more refined details in high-resolution images. In addition, APT enables a shortcut denoising process, resulting in faster sampling with minimal quality degradation. Our experimental results confirm that APT produces more detailed outputs with improved inference speed, providing a practical approach to high-resolution image generation.