Training Invertible Neural Networks as Autoencoders
This work addresses the potential for improved representation learning in unsupervised tasks, though it appears incremental as it builds on existing autoencoder and INN methods.
The authors tackled the problem of training invertible neural networks (INNs) as autoencoders, achieving results similar to classical autoencoders for low bottleneck sizes and outperforming them for large bottleneck sizes on datasets like MNIST, CIFAR, and CelebA.
Autoencoders are able to learn useful data representations in an unsupervised matter and have been widely used in various machine learning and computer vision tasks. In this work, we present methods to train Invertible Neural Networks (INNs) as (variational) autoencoders which we call INN (variational) autoencoders. Our experiments on MNIST, CIFAR and CelebA show that for low bottleneck sizes our INN autoencoder achieves results similar to the classical autoencoder. However, for large bottleneck sizes our INN autoencoder outperforms its classical counterpart. Based on the empirical results, we hypothesize that INN autoencoders might not have any intrinsic information loss and thereby are not bounded to a maximal number of layers (depth) after which only suboptimal results can be achieved.