Scalable Deep Unsupervised Clustering with Concrete GMVAEs
This work addresses a scalability bottleneck for researchers and practitioners using probabilistic clustering models, though it is incremental as it builds on existing VAE variants.
The paper tackles the high training time complexity of VAEs with discrete latent variables for clustering by applying a continuous relaxation, achieving a reduction from linear to constant complexity in the number of clusters. This approach reduced training time on CIFAR-100 with 20 clusters from 47 hours to less than 6 hours without compromising clustering quality.
Discrete random variables are natural components of probabilistic clustering models. A number of VAE variants with discrete latent variables have been developed. Training such methods requires marginalizing over the discrete latent variables, causing training time complexity to be linear in the number clusters. By applying a continuous relaxation to the discrete variables in these methods we can achieve a reduction in the training time complexity to be constant in the number of clusters used. We demonstrate that in practice for one such method, the Gaussian Mixture VAE, the use of a continuous relaxation has no negative effect on the quality of the clustering but provides a substantial reduction in training time, reducing training time on CIFAR-100 with 20 clusters from 47 hours to less than 6 hours.