Gradual training of deep denoising auto encoders
This incremental improvement addresses training efficiency for deep denoising autoencoders on mid-sized datasets.
The paper tackles the problem of training deep denoising autoencoders more effectively by proposing a gradual training scheme where layers are added and adapted incrementally. The result shows a small but consistent improvement in reconstruction quality and classification error on MNIST and CIFAR datasets compared to stacked training.
Stacked denoising auto encoders (DAEs) are well known to learn useful deep representations, which can be used to improve supervised training by initializing a deep network. We investigate a training scheme of a deep DAE, where DAE layers are gradually added and keep adapting as additional layers are added. We show that in the regime of mid-sized datasets, this gradual training provides a small but consistent improvement over stacked training in both reconstruction quality and classification error over stacked training on MNIST and CIFAR datasets.