LSD-C: Linearly Separable Deep Clusters
This addresses the challenge of unsupervised clustering for researchers and practitioners in machine learning, offering a novel approach that combines clustering with self-supervised techniques, though it appears incremental by building on existing semi-supervised learning practices.
The paper tackles the problem of identifying clusters in unlabeled datasets by proposing LSD-C, a method that enforces linear separation between clusters using pairwise connections and a binary cross-entropy loss, resulting in significant performance improvements over competitors on benchmarks like CIFAR 10/100, STL 10, MNIST, and Reuters 10K.
We present LSD-C, a novel method to identify clusters in an unlabeled dataset. Our algorithm first establishes pairwise connections in the feature space between the samples of the minibatch based on a similarity metric. Then it regroups in clusters the connected samples and enforces a linear separation between clusters. This is achieved by using the pairwise connections as targets together with a binary cross-entropy loss on the predictions that the associated pairs of samples belong to the same cluster. This way, the feature representation of the network will evolve such that similar samples in this feature space will belong to the same linearly separated cluster. Our method draws inspiration from recent semi-supervised learning practice and proposes to combine our clustering algorithm with self-supervised pretraining and strong data augmentation. We show that our approach significantly outperforms competitors on popular public image benchmarks including CIFAR 10/100, STL 10 and MNIST, as well as the document classification dataset Reuters 10K.