A Wasserstein Minimum Velocity Approach to Learning Unnormalized Models
This work addresses a computational bottleneck in machine learning for researchers and practitioners using unnormalized models, offering a scalable solution with potential domain-specific impacts.
The paper tackles the scalability issue of score matching for learning unnormalized models by introducing a scalable approximation based on a connection to Wasserstein gradient flows, resulting in applications for neural density estimators and auto-encoders with manifold-valued priors.
Score matching provides an effective approach to learning flexible unnormalized models, but its scalability is limited by the need to evaluate a second-order derivative. In this paper, we present a scalable approximation to a general family of learning objectives including score matching, by observing a new connection between these objectives and Wasserstein gradient flows. We present applications with promise in learning neural density estimators on manifolds, and training implicit variational and Wasserstein auto-encoders with a manifold-valued prior.