Filtered Posterior Mean Collections: A Unified Framework for Analytical Models of Diffusion Generalization
For researchers studying diffusion model generalization, this provides a unified framework and demonstrates incremental improvements over prior patch-based methods.
This work unifies existing analytical models of diffusion model generalization into a class called Filtered Posterior Mean Collections (FPMCs), and shows that performance can be improved via soft relaxations and source distribution augmentations, achieving consistent sample improvement across three natural image datasets.
The neural-network denoising functions which form the backbone of image diffusion models are remarkably consistent in their generalization behaviour across a wide variety of network architectures and training procedure hyperparameters. A recent line of research has sought to model the outputs of these networks by aggregating posterior weighted averages of training dataset patches. In this work, we consolidate these approaches into a unified model class which we call Filtered Posterior Mean Collections (FPMCs). We define this model class using query precision vectors, response weights, and source distributions, and illustrate that existing methods are recoverable with specific choices of these design axes. Investigating each axis in turn, we find that FPMC performance can be improved with soft relaxations of prior patch-based methods, and through augmentations of source distributions. Applying these findings to an existing FPMC, we demonstrate consistent sample improvement across three natural image datasets.