Latent Space Super-Resolution for Higher-Resolution Image Generation with Diffusion Models
This work addresses a specific bottleneck in image generation for applications requiring high-resolution outputs, representing an incremental improvement over existing reference-based methods.
The paper tackles the problem of generating high-resolution images (exceeding 1K) with diffusion models, which often suffer from structural distortions or content repetition at larger scales, and proposes LSRNA, a framework that combines latent space super-resolution and region-wise noise addition to outperform state-of-the-art reference-based methods in preserving detail and sharpness.
In this paper, we propose LSRNA, a novel framework for higher-resolution (exceeding 1K) image generation using diffusion models by leveraging super-resolution directly in the latent space. Existing diffusion models struggle with scaling beyond their training resolutions, often leading to structural distortions or content repetition. Reference-based methods address the issues by upsampling a low-resolution reference to guide higher-resolution generation. However, they face significant challenges: upsampling in latent space often causes manifold deviation, which degrades output quality. On the other hand, upsampling in RGB space tends to produce overly smoothed outputs. To overcome these limitations, LSRNA combines Latent space Super-Resolution (LSR) for manifold alignment and Region-wise Noise Addition (RNA) to enhance high-frequency details. Our extensive experiments demonstrate that integrating LSRNA outperforms state-of-the-art reference-based methods across various resolutions and metrics, while showing the critical role of latent space upsampling in preserving detail and sharpness. The code is available at https://github.com/3587jjh/LSRNA.