Wavelets to the Rescue: Improving Sample Quality of Latent Variable Deep Generative Models
This work addresses image quality issues in VAEs for computer vision applications, offering an incremental improvement by integrating wavelet decomposition into the generative process.
The paper tackles the problem of blurry image generation in Variational Autoencoders (VAEs) by proposing a wavelet space VAE that models images in the wavelet coefficient space, resulting in higher-quality images with consistently better FID scores than VAE architectures and competitive scores with GAN models on benchmark datasets.
Variational Autoencoders (VAE) are probabilistic deep generative models underpinned by elegant theory, stable training processes, and meaningful manifold representations. However, they produce blurry images due to a lack of explicit emphasis over high-frequency textural details of the images, and the difficulty to directly model the complex joint probability distribution over the high-dimensional image space. In this work, we approach these two challenges with a novel wavelet space VAE that uses the decoder to model the images in the wavelet coefficient space. This enables the VAE to emphasize over high-frequency components within an image obtained via wavelet decomposition. Additionally, by decomposing the complex function of generating high-dimensional images into inverse wavelet transformation and generation of wavelet coefficients, the latter becomes simpler to model by the VAE. We empirically validate that deep generative models operating in the wavelet space can generate images of higher quality than the image (RGB) space counterparts. Quantitatively, on benchmark natural image datasets, we achieve consistently better FID scores than VAE based architectures and competitive FID scores with a variety of GAN models for the same architectural and experimental setup. Furthermore, the proposed wavelet-based generative model retains desirable attributes like disentangled and informative latent representation without losing the quality in the generated samples.