Distribution Matching via Generalized Consistency Models
This work addresses distribution matching problems such as latent variable modeling and domain adaptation for researchers and practitioners in machine learning, offering a more stable alternative to GANs.
The paper tackles the challenge of distribution matching in generative models by proposing a novel approach inspired by consistency models from Continuous Normalizing Flows, which avoids the training difficulties of GANs like mode collapse and bi-level optimization, and demonstrates its performance on synthetic and real-world datasets.
Recent advancement in generative models have demonstrated remarkable performance across various data modalities. Beyond their typical use in data synthesis, these models play a crucial role in distribution matching tasks such as latent variable modeling, domain translation, and domain adaptation. Generative Adversarial Networks (GANs) have emerged as the preferred method of distribution matching due to their efficacy in handling high-dimensional data and their flexibility in accommodating various constraints. However, GANs often encounter challenge in training due to their bi-level min-max optimization objective and susceptibility to mode collapse. In this work, we propose a novel approach for distribution matching inspired by the consistency models employed in Continuous Normalizing Flow (CNF). Our model inherits the advantages of CNF models, such as having a straight forward norm minimization objective, while remaining adaptable to different constraints similar to GANs. We provide theoretical validation of our proposed objective and demonstrate its performance through experiments on synthetic and real-world datasets.