Sliced-Wasserstein normalizing flows: beyond maximum likelihood training
This addresses deficiencies in normalizing flows for generative modeling, offering incremental improvements in data fidelity and out-of-distribution detection.
The paper tackled the problem of normalizing flows generating unrealistic data and failing to detect out-of-distribution data by proposing a hybrid training objective combining maximum likelihood and sliced-Wasserstein distance, resulting in better generative abilities with improved likelihood and visual quality on synthetic and real image datasets, and lower likelihood for out-of-distribution data.
Despite their advantages, normalizing flows generally suffer from several shortcomings including their tendency to generate unrealistic data (e.g., images) and their failing to detect out-of-distribution data. One reason for these deficiencies lies in the training strategy which traditionally exploits a maximum likelihood principle only. This paper proposes a new training paradigm based on a hybrid objective function combining the maximum likelihood principle (MLE) and a sliced-Wasserstein distance. Results obtained on synthetic toy examples and real image data sets show better generative abilities in terms of both likelihood and visual aspects of the generated samples. Reciprocally, the proposed approach leads to a lower likelihood of out-of-distribution data, demonstrating a greater data fidelity of the resulting flows.