From colouring-in to pointillism: revisiting semantic segmentation supervision
This addresses the scalability issue in semantic segmentation annotation for researchers and practitioners, offering a more efficient alternative to traditional dense labeling methods.
The paper tackles the bottleneck of dense pixel-level annotation in semantic segmentation by proposing a pointillist approach using point-wise yes/no questions, demonstrating its effectiveness by collecting 22.6M point labels over 4,171 classes on the Open Images dataset.
The prevailing paradigm for producing semantic segmentation training data relies on densely labelling each pixel of each image in the training set, akin to colouring-in books. This approach becomes a bottleneck when scaling up in the number of images, classes, and annotators. Here we propose instead a pointillist approach for semantic segmentation annotation, where only point-wise yes/no questions are answered. We explore design alternatives for such an active learning approach, measure the speed and consistency of human annotators on this task, show that this strategy enables training good segmentation models, and that it is suitable for evaluating models at test time. As concrete proof of the scalability of our method, we collected and released 22.6M point labels over 4,171 classes on the Open Images dataset. Our results enable to rethink the semantic segmentation pipeline of annotation, training, and evaluation from a pointillism point of view.