Enhancing diffusion models with Gaussianization preprocessing
This work addresses a bottleneck in diffusion models for generative tasks, offering a method to enhance efficiency and stability, though it appears incremental as it builds on existing preprocessing techniques.
The paper tackles the slow sampling and early-stage generation quality issues in diffusion models by applying Gaussianization preprocessing to training data, which improves reconstruction quality, especially for small-scale networks.
Diffusion models are a class of generative models that have demonstrated remarkable success in tasks such as image generation. However, one of the bottlenecks of these models is slow sampling due to the delay before the onset of trajectory bifurcation, at which point substantial reconstruction begins. This issue degrades generation quality, especially in the early stages. Our primary objective is to mitigate bifurcation-related issues by preprocessing the training data to enhance reconstruction quality, particularly for small-scale network architectures. Specifically, we propose applying Gaussianization preprocessing to the training data to make the target distribution more closely resemble an independent Gaussian distribution, which serves as the initial density of the reconstruction process. This preprocessing step simplifies the model's task of learning the target distribution, thereby improving generation quality even in the early stages of reconstruction with small networks. The proposed method is, in principle, applicable to a broad range of generative tasks, enabling more stable and efficient sampling processes.