Discrete Denoising Flows
This work addresses the challenge of efficient and accurate generative modeling for discrete data, which is incremental as it builds upon existing discrete flow-based models with a novel training approach.
The paper tackles the problem of generative modeling for discrete data by introducing Discrete Denoising Flows (DDFs), a new discrete flow-based model that enables local training without gradient bias and outperforms existing Discrete Flows in log-likelihood on tasks including binary MNIST and Cityscapes segmentation maps.
Discrete flow-based models are a recently proposed class of generative models that learn invertible transformations for discrete random variables. Since they do not require data dequantization and maximize an exact likelihood objective, they can be used in a straight-forward manner for lossless compression. In this paper, we introduce a new discrete flow-based model for categorical random variables: Discrete Denoising Flows (DDFs). In contrast with other discrete flow-based models, our model can be locally trained without introducing gradient bias. We show that DDFs outperform Discrete Flows on modeling a toy example, binary MNIST and Cityscapes segmentation maps, measured in log-likelihood.