Generalization Bounds For Unsupervised and Semi-Supervised Learning With Autoencoders
This work addresses a theoretical gap for researchers in machine learning, offering foundational insights that could enhance semi-supervised learning schemes, though it is incremental in building on existing generalization theory.
The paper tackles the lack of theoretical understanding of autoencoders' generalization properties in unsupervised and semi-supervised learning by providing the first generalization bounds for autoencoders using a novel reconstruction loss and recent deep learning theory, supported by empirical demonstrations.
Autoencoders are widely used for unsupervised learning and as a regularization scheme in semi-supervised learning. However, theoretical understanding of their generalization properties and of the manner in which they can assist supervised learning has been lacking. We utilize recent advances in the theory of deep learning generalization, together with a novel reconstruction loss, to provide generalization bounds for autoencoders. To the best of our knowledge, this is the first such bound. We further show that, under appropriate assumptions, an autoencoder with good generalization properties can improve any semi-supervised learning scheme. We support our theoretical results with empirical demonstrations.