A Theoretical Analysis of Discrete Flow Matching Generative Models
This work offers foundational theoretical guarantees for a discrete generative modeling framework, addressing a gap in formal convergence proofs for such models.
The paper provides a theoretical analysis of Discrete Flow Matching (DFM) generative models, proving that the generated distribution converges to the true data distribution as training set size increases, with error bounds based on approximation and estimation errors.
We provide a theoretical analysis for end-to-end training Discrete Flow Matching (DFM) generative models. DFM is a promising discrete generative modeling framework that learns the underlying generative dynamics by training a neural network to approximate the transformative velocity field. Our analysis establishes a clear chain of guarantees by decomposing the final distribution estimation error. We first prove that the total variation distance between the generated and target distributions is controlled by the risk of the learned velocity field. We then bound this risk by analyzing its two primary sources: (i) Approximation Error, where we quantify the capacity of the Transformer architecture to represent the true velocity, and (ii) Estimation Error, where we derive statistical convergence rates that bound the error from training on a finite dataset. By composing these results, we provide the first formal proof that the distribution generated by a trained DFM model provably converges to the true data distribution as the training set size increases.