Laplacian Denoising Autoencoder
This work addresses the need for scalable unsupervised learning frameworks to reduce labeling costs in machine learning, though it is incremental as it builds on existing denoising autoencoder approaches.
The paper tackles the problem of learning robust representations from unlabeled data by proposing a Laplacian denoising autoencoder that corrupts input data in the gradient domain across multiple scales, resulting in better representations compared to single-scale and other methods, as demonstrated on visual benchmarks.
While deep neural networks have been shown to perform remarkably well in many machine learning tasks, labeling a large amount of ground truth data for supervised training is usually very costly to scale. Therefore, learning robust representations with unlabeled data is critical in relieving human effort and vital for many downstream tasks. Recent advances in unsupervised and self-supervised learning approaches for visual data have benefited greatly from domain knowledge. Here we are interested in a more generic unsupervised learning framework that can be easily generalized to other domains. In this paper, we propose to learn data representations with a novel type of denoising autoencoder, where the noisy input data is generated by corrupting latent clean data in the gradient domain. This can be naturally generalized to span multiple scales with a Laplacian pyramid representation of the input data. In this way, the agent learns more robust representations that exploit the underlying data structures across multiple scales. Experiments on several visual benchmarks demonstrate that better representations can be learned with the proposed approach, compared to its counterpart with single-scale corruption and other approaches. Furthermore, we also demonstrate that the learned representations perform well when transferring to other downstream vision tasks.