Revisiting Self-Supervised Visual Representation Learning
This work addresses the problem of improving unsupervised visual representation learning for computer vision researchers, but it appears incremental as it builds on existing techniques.
The authors revisited self-supervised visual representation learning, challenging common practices and CNN design standards, and significantly boosted performance to outperform previous state-of-the-art results by a large margin.
Unsupervised visual representation learning remains a largely unsolved problem in computer vision research. Among a big body of recently proposed approaches for unsupervised learning of visual representations, a class of self-supervised techniques achieves superior performance on many challenging benchmarks. A large number of the pretext tasks for self-supervised learning have been studied, but other important aspects, such as the choice of convolutional neural networks (CNN), has not received equal attention. Therefore, we revisit numerous previously proposed self-supervised models, conduct a thorough large scale study and, as a result, uncover multiple crucial insights. We challenge a number of common practices in selfsupervised visual representation learning and observe that standard recipes for CNN design do not always translate to self-supervised representation learning. As part of our study, we drastically boost the performance of previously proposed techniques and outperform previously published state-of-the-art results by a large margin.