On tuning consistent annealed sampling for denoising score matching
This work addresses a specific tuning challenge for researchers and practitioners using denoising score matching, but it is incremental as it builds on existing sampling methods without introducing a new paradigm.
The paper tackles the problem of tuning hyperparameters for consistent annealed sampling in score-based generative models, proposing a formulation that facilitates tuning with few or variable steps and showing connections to other sampling schemes.
Score-based generative models provide state-of-the-art quality for image and audio synthesis. Sampling from these models is performed iteratively, typically employing a discretized series of noise levels and a predefined scheme. In this note, we first overview three common sampling schemes for models trained with denoising score matching. Next, we focus on one of them, consistent annealed sampling, and study its hyper-parameter boundaries. We then highlight a possible formulation of such hyper-parameter that explicitly considers those boundaries and facilitates tuning when using few or a variable number of steps. Finally, we highlight some connections of the formulation with other sampling schemes.