Uncertainty Autoencoders: Learning Compressed Representations via Variational Information Maximization
This work addresses the challenge of efficient data acquisition and recovery in compressed sensing, offering a unified framework that could benefit applications in dimensionality reduction and generative modeling, though it appears incremental by building on existing autoencoder and variational methods.
The paper tackles the problem of learning compressed representations for high-dimensional data by proposing Uncertainty Autoencoders, which optimize a variational lower bound to mutual information, resulting in a 32% average improvement over competing approaches for statistical compressed sensing.
Compressed sensing techniques enable efficient acquisition and recovery of sparse, high-dimensional data signals via low-dimensional projections. In this work, we propose Uncertainty Autoencoders, a learning framework for unsupervised representation learning inspired by compressed sensing. We treat the low-dimensional projections as noisy latent representations of an autoencoder and directly learn both the acquisition (i.e., encoding) and amortized recovery (i.e., decoding) procedures. Our learning objective optimizes for a tractable variational lower bound to the mutual information between the datapoints and the latent representations. We show how our framework provides a unified treatment to several lines of research in dimensionality reduction, compressed sensing, and generative modeling. Empirically, we demonstrate a 32% improvement on average over competing approaches for the task of statistical compressed sensing of high-dimensional datasets.