Gradient-Free Score-Based Sampling Methods with Ensembles
This provides a practical solution for scientific and engineering applications where gradient computation is prohibitive, though it is an incremental advancement building on existing score-based and ensemble techniques.
The paper tackled the problem of sampling from complex probability distributions when gradients are unavailable or too costly, by introducing gradient-free ensemble methods within score-based sampling, and demonstrated efficacy on multi-modal and non-Gaussian distributions, including a high-dimensional geophysical application.
Recent developments in generative modeling have utilized score-based methods coupled with stochastic differential equations to sample from complex probability distributions. However, these and other performant sampling methods generally require gradients of the target probability distribution, which can be unavailable or computationally prohibitive in many scientific and engineering applications. Here, we introduce ensembles within score-based sampling methods to develop gradient-free approximate sampling techniques that leverage the collective dynamics of particle ensembles to compute approximate reverse diffusion drifts. We introduce the underlying methodology, emphasizing its relationship with generative diffusion models and the previously introduced Föllmer sampler. We demonstrate the efficacy of the ensemble strategies through various examples, ranging from low- to medium-dimensionality sampling problems, including multi-modal and highly non-Gaussian probability distributions, and provide comparisons to traditional methods like the No-U-Turn Sampler. Additionally, we showcase these strategies in the context of a high-dimensional Bayesian inversion problem within the geophysical sciences. Our findings highlight the potential of ensemble strategies for modeling complex probability distributions in situations where gradients are unavailable.