Continuous normalizing flows on manifolds
This work addresses a fundamental problem in machine learning for researchers and practitioners dealing with data on complex geometric spaces, representing a novel method rather than an incremental improvement.
The paper tackles the limitation of normalizing flows to basic geometries by extending continuous normalizing flows to arbitrary smooth manifolds using differential geometry and geometric control theory, enabling reparameterizable sampling from complex distributions on nontrivial topologies.
Normalizing flows are a powerful technique for obtaining reparameterizable samples from complex multimodal distributions. Unfortunately, current approaches are only available for the most basic geometries and fall short when the underlying space has a nontrivial topology, limiting their applicability for most real-world data. Using fundamental ideas from differential geometry and geometric control theory, we describe how the recently introduced Neural ODEs and continuous normalizing flows can be extended to arbitrary smooth manifolds. We propose a general methodology for parameterizing vector fields on these spaces and demonstrate how gradient-based learning can be performed. Additionally, we provide a scalable unbiased estimator for the divergence in this generalized setting. Experiments on a diverse selection of spaces empirically showcase the defined framework's ability to obtain reparameterizable samples from complex distributions.