Lookahead optimizer improves the performance of Convolutional Autoencoders for reconstruction of natural images
This work provides an incremental improvement for researchers and practitioners working with autoencoders for image reconstruction tasks.
This paper investigates the use of the Lookahead optimizer with Convolutional Autoencoders (CAE) and Convolutional Variational Autoencoders (CVAE) for image reconstruction. The study found that the Lookahead optimizer, when combined with Adam, improved the reconstruction performance of CAEs on a movie dataset of natural images.
Autoencoders are a class of artificial neural networks which have gained a lot of attention in the recent past. Using the encoder block of an autoencoder the input image can be compressed into a meaningful representation. Then a decoder is employed to reconstruct the compressed representation back to a version which looks like the input image. It has plenty of applications in the field of data compression and denoising. Another version of Autoencoders (AE) exist, called Variational AE (VAE) which acts as a generative model like GAN. Recently, an optimizer was introduced which is known as lookahead optimizer which significantly enhances the performances of Adam as well as SGD. In this paper, we implement Convolutional Autoencoders (CAE) and Convolutional Variational Autoencoders (CVAE) with lookahead optimizer (with Adam) and compare them with the Adam (only) optimizer counterparts. For this purpose, we have used a movie dataset comprising of natural images for the former case and CIFAR100 for the latter case. We show that lookahead optimizer (with Adam) improves the performance of CAEs for reconstruction of natural images.