Generative Modeling of Discrete Joint Distributions by E-Geodesic Flow Matching on Assignment Manifolds
This addresses the problem of generating structured discrete data with complex dependencies for machine learning applications, representing a novel method for a known bottleneck.
The paper tackles generative modeling of discrete joint distributions by developing a continuous normalizing flow method on assignment manifolds, which avoids discretization issues like rounding and truncation. Experiments demonstrate the approach's broad applicability.
This paper introduces a novel generative model for discrete distributions based on continuous normalizing flows on the submanifold of factorizing discrete measures. Integration of the flow gradually assigns categories and avoids issues of discretizing the latent continuous model like rounding, sample truncation etc. General non-factorizing discrete distributions capable of representing complex statistical dependencies of structured discrete data, can be approximated by embedding the submanifold into a the meta-simplex of all joint discrete distributions and data-driven averaging. Efficient training of the generative model is demonstrated by matching the flow of geodesics of factorizing discrete distributions. Various experiments underline the approach's broad applicability.