Self-supervised Learning with Fully Convolutional Networks
This work provides an incremental improvement in self-supervised learning for semantic segmentation, which could benefit researchers and practitioners dealing with limited annotated data.
This paper addresses the challenge of learning representations from unlabeled data for semantic segmentation. By formulating the Jigsaw Puzzle problem as a patch-wise classification task and solving it with a fully convolutional network, the authors achieved a 5.8 percentage point improvement over a randomly initialized baseline on the Cityscapes dataset.
Although deep learning based methods have achieved great success in many computer vision tasks, their performance relies on a large number of densely annotated samples that are typically difficult to obtain. In this paper, we focus on the problem of learning representation from unlabeled data for semantic segmentation. Inspired by two patch-based methods, we develop a novel self-supervised learning framework by formulating the Jigsaw Puzzle problem as a patch-wise classification process and solving it with a fully convolutional network. By learning to solve a Jigsaw Puzzle problem with 25 patches and transferring the learned features to semantic segmentation task on Cityscapes dataset, we achieve a 5.8 percentage point improvement over the baseline model that initialized from random values. Moreover, experiments show that our self-supervised learning method can be applied to different datasets and models. In particular, we achieved competitive performance with the state-of-the-art methods on the PASCAL VOC2012 dataset using significant fewer training images.