SuperCM: Improving Semi-Supervised Learning and Domain Adaptation through differentiable clustering
This work addresses the challenge of limited labeled data for machine learning practitioners, though it is incremental as it builds on existing clustering assumptions.
The paper tackles the problem of improving semi-supervised learning and domain adaptation by explicitly using a differentiable clustering module that leverages supervised data to compute centroids, resulting in enhanced performance, especially in low supervision regimes, as demonstrated through extensive experiments.
Semi-Supervised Learning (SSL) and Unsupervised Domain Adaptation (UDA) enhance the model performance by exploiting information from labeled and unlabeled data. The clustering assumption has proven advantageous for learning with limited supervision and states that data points belonging to the same cluster in a high-dimensional space should be assigned to the same category. Recent works have utilized different training mechanisms to implicitly enforce this assumption for the SSL and UDA. In this work, we take a different approach by explicitly involving a differentiable clustering module which is extended to leverage the supervised data to compute its centroids. We demonstrate the effectiveness of our straightforward end-to-end training strategy for SSL and UDA over extensive experiments and highlight its benefits, especially in low supervision regimes, both as a standalone model and as a regularizer for existing approaches.