Adaptive Compression of the Latent Space in Variational Autoencoders
This incremental improvement addresses a known bottleneck in VAEs for researchers and practitioners by simplifying hyperparameter tuning.
The paper tackles the challenge of determining the optimal latent space size in Variational Autoencoders by introducing a method that automatically adjusts it during training through neuron removal, achieving results comparable to optimal grid search while being significantly faster on four image datasets.
Variational Autoencoders (VAEs) are powerful generative models that have been widely used in various fields, including image and text generation. However, one of the known challenges in using VAEs is the model's sensitivity to its hyperparameters, such as the latent space size. This paper presents a simple extension of VAEs for automatically determining the optimal latent space size during the training process by gradually decreasing the latent size through neuron removal and observing the model performance. The proposed method is compared to traditional hyperparameter grid search and is shown to be significantly faster while still achieving the best optimal dimensionality on four image datasets. Furthermore, we show that the final performance of our method is comparable to training on the optimal latent size from scratch, and might thus serve as a convenient substitute.