Deep clustering using adversarial net based clustering loss
This work addresses a bottleneck in deep clustering for unsupervised learning, though it is incremental as it builds on existing adversarial and clustering techniques.
The paper tackles the constraint of requiring a closed-form loss function in deep clustering by reformulating it as an adversarial network using KL divergence, achieving on-par or better performance on datasets like MNIST, REUTERS10K, and CIFAR10 compared to state-of-the-art methods.
Deep clustering is a recent deep learning technique which combines deep learning with traditional unsupervised clustering. At the heart of deep clustering is a loss function which penalizes samples for being an outlier from their ground truth cluster centers in the latent space. The probabilistic variant of deep clustering reformulates the loss using KL divergence. Often, the main constraint of deep clustering is the necessity of a closed form loss function to make backpropagation tractable. Inspired by deep clustering and adversarial net, we reformulate deep clustering as an adversarial net over traditional closed form KL divergence. Training deep clustering becomes a task of minimizing the encoder and maximizing the discriminator. At optimality, this method theoretically approaches the JS divergence between the distribution assumption of the encoder and the discriminator. We demonstrated the performance of our proposed method on several well cited datasets such as MNIST, REUTERS10K and CIFAR10, achieving on-par or better performance with some of the state-of-the-art deep clustering methods.