Data Transformation Insights in Self-supervision with Clustering Tasks
This work addresses a gap in self-supervised learning for label-scarce domains by analyzing transformation effects, though it appears incremental as it builds on existing methods without introducing a new paradigm.
The paper tackles the problem of understanding the impact of data transformations in self-supervised clustering tasks, showing that certain transformations improve convergence rates while others can be harmful, with empirical results supporting theoretical insights.
Self-supervision is key to extending use of deep learning for label scarce domains. For most of self-supervised approaches data transformations play an important role. However, up until now the impact of transformations have not been studied. Furthermore, different transformations may have different impact on the system. We provide novel insights into the use of data transformation in self-supervised tasks, specially pertaining to clustering. We show theoretically and empirically that certain set of transformations are helpful in convergence of self-supervised clustering. We also show the cases when the transformations are not helpful or in some cases even harmful. We show faster convergence rate with valid transformations for convex as well as certain family of non-convex objectives along with the proof of convergence to the original set of optima. We have synthetic as well as real world data experiments. Empirically our results conform with the theoretical insights provided.