Sampling in Unit Time with Kernel Fisher-Rao Flow
This work addresses the challenge of efficient sampling in machine learning and statistics, offering a gradient-free approach that could benefit applications where gradient computation is costly or infeasible, though it appears incremental as it builds on existing frameworks like Fisher-Rao gradient flow and optimal transport.
The authors tackled the problem of sampling from unnormalized target densities by introducing a new mean-field ODE and interacting particle systems that are gradient-free and rely on sampling from a reference density and computing density ratios. They demonstrated empirically that their method produces high-quality samples, outperforming comparable gradient-free systems and competing with gradient-based alternatives.
We introduce a new mean-field ODE and corresponding interacting particle systems (IPS) for sampling from an unnormalized target density. The IPS are gradient-free, available in closed form, and only require the ability to sample from a reference density and compute the (unnormalized) target-to-reference density ratio. The mean-field ODE is obtained by solving a Poisson equation for a velocity field that transports samples along the geometric mixture of the two densities, which is the path of a particular Fisher-Rao gradient flow. We employ a RKHS ansatz for the velocity field, which makes the Poisson equation tractable and enables discretization of the resulting mean-field ODE over finite samples. The mean-field ODE can be additionally be derived from a discrete-time perspective as the limit of successive linearizations of the Monge-Ampère equations within a framework known as sample-driven optimal transport. We introduce a stochastic variant of our approach and demonstrate empirically that our IPS can produce high-quality samples from varied target distributions, outperforming comparable gradient-free particle systems and competitive with gradient-based alternatives.