Generative Modeling with Bayesian Sample Inference
This work addresses generative modeling for AI researchers, offering an incremental improvement over existing methods like diffusion models.
The paper tackles generative modeling by deriving a novel approach from iterative Gaussian posterior inference, treating generated samples as unknown variables, and demonstrates improved sample quality on ImageNet32 over Bayesian Flow Networks and Variational Diffusion Models while achieving equivalent log-likelihoods on ImageNet32 and CIFAR10.
We derive a novel generative model from iterative Gaussian posterior inference. By treating the generated sample as an unknown variable, we can formulate the sampling process in the language of Bayesian probability. Our model uses a sequence of prediction and posterior update steps to iteratively narrow down the unknown sample starting from a broad initial belief. In addition to a rigorous theoretical analysis, we establish a connection between our model and diffusion models and show that it includes Bayesian Flow Networks (BFNs) as a special case. In our experiments, we demonstrate that our model improves sample quality on ImageNet32 over both BFNs and the closely related Variational Diffusion Models, while achieving equivalent log-likelihoods on ImageNet32 and CIFAR10. Find our code at https://github.com/martenlienen/bsi.