Online Semi-Supervised Learning with Deep Hybrid Boltzmann Machines and Denoising Autoencoders
This work addresses semi-supervised learning problems, particularly for data-streams in life-long learning, but appears incremental as it builds on existing deep learning methods.
The paper tackled semi-supervised learning by proposing two novel deep hybrid architectures, Deep Hybrid Boltzmann Machine and Deep Hybrid Denoising Auto-encoder, which showed improved performance compared to a baseline method.
Two novel deep hybrid architectures, the Deep Hybrid Boltzmann Machine and the Deep Hybrid Denoising Auto-encoder, are proposed for handling semi-supervised learning problems. The models combine experts that model relevant distributions at different levels of abstraction to improve overall predictive performance on discriminative tasks. Theoretical motivations and algorithms for joint learning for each are presented. We apply the new models to the domain of data-streams in work towards life-long learning. The proposed architectures show improved performance compared to a pseudo-labeled, drop-out rectifier network.