From Principal Subspaces to Principal Components with Linear Autoencoders
This solves a specific technical problem in unsupervised learning for researchers and practitioners using linear autoencoders.
The paper addresses the discrepancy between autoencoder weights and principal component loading vectors, demonstrating a method to recover the exact loading vectors from autoencoder weights.
The autoencoder is an effective unsupervised learning model which is widely used in deep learning. It is well known that an autoencoder with a single fully-connected hidden layer, a linear activation function and a squared error cost function trains weights that span the same subspace as the one spanned by the principal component loading vectors, but that they are not identical to the loading vectors. In this paper, we show how to recover the loading vectors from the autoencoder weights.