Generative Modeling through the Semi-dual Formulation of Unbalanced Optimal Transport
This work addresses generative modeling challenges for machine learning practitioners by providing a more robust and efficient method, though it appears incremental as it builds on OT with UOT relaxation.
The paper tackles the problem of generative modeling using Optimal Transport (OT), which is sensitive to outliers and has optimization challenges, by proposing a novel model based on the semi-dual formulation of Unbalanced Optimal Transport (UOT) to improve robustness, stability, and convergence. It achieves FID scores of 2.97 on CIFAR-10 and 6.36 on CelebA-HQ-256, outperforming existing OT-based models.
Optimal Transport (OT) problem investigates a transport map that bridges two distributions while minimizing a given cost function. In this regard, OT between tractable prior distribution and data has been utilized for generative modeling tasks. However, OT-based methods are susceptible to outliers and face optimization challenges during training. In this paper, we propose a novel generative model based on the semi-dual formulation of Unbalanced Optimal Transport (UOT). Unlike OT, UOT relaxes the hard constraint on distribution matching. This approach provides better robustness against outliers, stability during training, and faster convergence. We validate these properties empirically through experiments. Moreover, we study the theoretical upper-bound of divergence between distributions in UOT. Our model outperforms existing OT-based generative models, achieving FID scores of 2.97 on CIFAR-10 and 6.36 on CelebA-HQ-256. The code is available at \url{https://github.com/Jae-Moo/UOTM}.