SiamMM: A Mixture Model Perspective on Deep Unsupervised Learning
It addresses the unclear optimal methodology in unsupervised learning for researchers, though it appears incremental by enhancing existing clustering methods.
The paper tackled the heuristic application of clustering in self-supervised learning by connecting it to classical mixture models, resulting in SiamMM, which achieved state-of-the-art performance on benchmarks and revealed potential mislabeling in data.
Recent studies have demonstrated the effectiveness of clustering-based approaches for self-supervised and unsupervised learning. However, the application of clustering is often heuristic, and the optimal methodology remains unclear. In this work, we establish connections between these unsupervised clustering methods and classical mixture models from statistics. Through this framework, we demonstrate significant enhancements to these clustering methods, leading to the development of a novel model named SiamMM. Our method attains state-of-the-art performance across various self-supervised learning benchmarks. Inspection of the learned clusters reveals a strong resemblance to unseen ground truth labels, uncovering potential instances of mislabeling.