Training Generative Networks with general Optimal Transport distances
This work addresses the challenge of flexible and stable training of generative models for machine learning applications, representing an incremental improvement over existing methods like WGANs.
The authors tackled the problem of training generative networks by proposing a new algorithm that uses an auxiliary neural network to express the potential of the optimal transport map between data distributions, enabling explicit use of any transportation cost function, such as the squared distance for Wasserstein-2 metric, resulting in fast and stable gradient descents.
We propose a new algorithm that uses an auxiliary neural network to express the potential of the optimal transport map between two data distributions. In the sequel, we use the aforementioned map to train generative networks. Unlike WGANs, where the Euclidean distance is ${\it implicitly}$ used, this new method allows to ${\it explicitly}$ use ${\it any}$ transportation cost function that can be chosen to match the problem at hand. For example, it allows to use the squared distance as a transportation cost function, giving rise to the Wasserstein-2 metric for probability distributions, which results in fast and stable gradient descends. It also allows to use image centered distances, like the structure similarity index, with notable differences in the results.