Coulomb Autoencoders
This work addresses a fundamental challenge in generative modeling for machine learning practitioners, offering theoretical and empirical improvements, though it appears incremental as it builds on existing MMD-based methods.
The paper tackles the problem of learning true densities in high-dimensional spaces by proposing Coulomb autoencoders based on maximum-mean discrepancy (MMD) with Coulomb kernels, proving optimal convergence and generalization bounds, and showing state-of-the-art performance on synthetic data and a celebrity faces dataset.
Learning the true density in high-dimensional feature spaces is a well-known problem in machine learning. In this work, we consider generative autoencoders based on maximum-mean discrepancy (MMD) and provide theoretical insights. In particular, (i) we prove that MMD coupled with Coulomb kernels has optimal convergence properties, which are similar to convex functionals, thus improving the training of autoencoders, and (ii) we provide a probabilistic bound on the generalization performance, highlighting some fundamental conditions to achieve better generalization. We validate the theory on synthetic examples and on the popular dataset of celebrities' faces, showing that our model, called Coulomb autoencoders, outperform the state-of-the-art.