Image Neural Field Diffusion Models
This work addresses a limitation in diffusion models for image generation, enabling more flexible and efficient handling of multi-resolution data and inverse problems, though it builds incrementally on existing techniques.
The authors tackled the problem of diffusion models being limited to fixed-resolution images by proposing to learn the distribution of continuous images using image neural fields, which can be rendered at any resolution, and showed that this approach outperforms fixed-resolution models followed by super-resolution models.
Diffusion models have shown an impressive ability to model complex data distributions, with several key advantages over GANs, such as stable training, better coverage of the training distribution's modes, and the ability to solve inverse problems without extra training. However, most diffusion models learn the distribution of fixed-resolution images. We propose to learn the distribution of continuous images by training diffusion models on image neural fields, which can be rendered at any resolution, and show its advantages over fixed-resolution models. To achieve this, a key challenge is to obtain a latent space that represents photorealistic image neural fields. We propose a simple and effective method, inspired by several recent techniques but with key changes to make the image neural fields photorealistic. Our method can be used to convert existing latent diffusion autoencoders into image neural field autoencoders. We show that image neural field diffusion models can be trained using mixed-resolution image datasets, outperform fixed-resolution diffusion models followed by super-resolution models, and can solve inverse problems with conditions applied at different scales efficiently.