A Statistical Analysis of Wasserstein Autoencoders for Intrinsically Low-dimensional Data
This provides theoretical foundations for WAEs in practical applications like image processing, though it is incremental as it builds on existing VAE/WAE frameworks.
The paper tackles the lack of statistical guarantees for Wasserstein Autoencoders (WAEs) on intrinsically low-dimensional data, showing that with proper network architectures, WAEs can learn data distributions with convergence rates dependent only on the intrinsic dimension, not the high feature dimension.
Variational Autoencoders (VAEs) have gained significant popularity among researchers as a powerful tool for understanding unknown distributions based on limited samples. This popularity stems partly from their impressive performance and partly from their ability to provide meaningful feature representations in the latent space. Wasserstein Autoencoders (WAEs), a variant of VAEs, aim to not only improve model efficiency but also interpretability. However, there has been limited focus on analyzing their statistical guarantees. The matter is further complicated by the fact that the data distributions to which WAEs are applied - such as natural images - are often presumed to possess an underlying low-dimensional structure within a high-dimensional feature space, which current theory does not adequately account for, rendering known bounds inefficient. To bridge the gap between the theory and practice of WAEs, in this paper, we show that WAEs can learn the data distributions when the network architectures are properly chosen. We show that the convergence rates of the expected excess risk in the number of samples for WAEs are independent of the high feature dimension, instead relying only on the intrinsic dimension of the data distribution.