Simplex-to-Euclidean Bijections for Categorical Flow Matching
This addresses the challenge of modeling categorical data for researchers in machine learning, offering an incremental improvement over existing methods that use Riemannian geometry or custom noise processes.
The paper tackles the problem of learning and sampling from probability distributions on the simplex by proposing a method that maps the simplex to Euclidean space using smooth bijections based on Aitchison geometry, enabling density modeling in Euclidean space while allowing exact recovery of discrete distributions. It achieves competitive performance on synthetic and real-world datasets.
We propose a method for learning and sampling from probability distributions supported on the simplex. Our approach maps the open simplex to Euclidean space via smooth bijections, leveraging the Aitchison geometry to define the mappings, and supports modeling categorical data by a Dirichlet interpolation that dequantizes discrete observations into continuous ones. This enables density modeling in Euclidean space through the bijection while still allowing exact recovery of the original discrete distribution. Compared to previous methods that operate on the simplex using Riemannian geometry or custom noise processes, our approach works in Euclidean space while respecting the Aitchison geometry, and achieves competitive performance on both synthetic and real-world data sets.