Importance Sampling via Score-based Generative Models
This work addresses the need for efficient importance sampling in real-world scenarios where multiple biased sampling tasks rely on a single base distribution, offering a novel, training-free solution.
The authors tackled the problem of importance sampling by proposing a training-free framework that uses score-based generative models (SGMs) as the base probability density function, eliminating the need for additional training. They demonstrated the method's scalability and effectiveness across diverse datasets, including industrial and natural images with neural importance weight functions.
Importance sampling, which involves sampling from a probability density function (PDF) proportional to the product of an importance weight function and a base PDF, is a powerful technique with applications in variance reduction, biased or customized sampling, data augmentation, and beyond. Inspired by the growing availability of score-based generative models (SGMs), we propose an entirely training-free Importance sampling framework that relies solely on an SGM for the base PDF. Our key innovation is realizing the importance sampling process as a backward diffusion process, expressed in terms of the score function of the base PDF and the specified importance weight function--both readily available--eliminating the need for any additional training. We conduct a thorough analysis demonstrating the method's scalability and effectiveness across diverse datasets and tasks, including importance sampling for industrial and natural images with neural importance weight functions. The training-free aspect of our method is particularly compelling in real-world scenarios where a single base distribution underlies multiple biased sampling tasks, each requiring a different importance weight function. To the best of our knowledge our approach is the first importance sampling framework to achieve this.