Auto-Encoding Goodness of Fit
This work addresses the challenge of balancing reconstruction and generation in autoencoders for machine learning applications, presenting an incremental improvement with a novel regularization approach.
The authors tackled the problem of improving generative autoencoders by incorporating goodness-of-fit tests at multiple levels, resulting in a model that achieves comparable FID scores and mean squared errors to competing models while maintaining statistical indistinguishability from Gaussian in the latent space.
We develop a new type of generative autoencoder called the Goodness-of-Fit Autoencoder (GoFAE), which incorporates GoF tests at two levels. At the minibatch level, it uses GoF test statistics as regularization objectives. At a more global level, it selects a regularization coefficient based on higher criticism, i.e., a test on the uniformity of the local GoF p-values. We justify the use of GoF tests by providing a relaxed $L_2$-Wasserstein bound on the distance between the latent distribution and a distribution class. We prove that optimization based on these tests can be done with stochastic gradient descent on a compact Riemannian manifold. Empirically, we show that our higher criticism parameter selection procedure balances reconstruction and generation using mutual information and uniformity of p-values respectively. Finally, we show that GoFAE achieves comparable FID scores and mean squared errors with competing deep generative models while retaining statistical indistinguishability from Gaussian in the latent space based on a variety of hypothesis tests.