Boosting Supervised Learning Performance with Co-training
This work addresses the labeling burden for companies or individuals in perception tasks, but it is incremental as it builds on existing self-supervised learning methods.
The paper tackles the problem of high labeling costs for deep learning perception models by proposing a lightweight self-supervised co-training framework that integrates pretext tasks into supervised tasks with minimal overhead. Results show improved accuracy in object detection and panoptic segmentation, with strong domain adaptation using unlabeled data.
Deep learning perception models require a massive amount of labeled training data to achieve good performance. While unlabeled data is easy to acquire, the cost of labeling is prohibitive and could create a tremendous burden on companies or individuals. Recently, self-supervision has emerged as an alternative to leveraging unlabeled data. In this paper, we propose a new light-weight self-supervised learning framework that could boost supervised learning performance with minimum additional computation cost. Here, we introduce a simple and flexible multi-task co-training framework that integrates a self-supervised task into any supervised task. Our approach exploits pretext tasks to incur minimum compute and parameter overheads and minimal disruption to existing training pipelines. We demonstrate the effectiveness of our framework by using two self-supervised tasks, object detection and panoptic segmentation, on different perception models. Our results show that both self-supervised tasks can improve the accuracy of the supervised task and, at the same time, demonstrates strong domain adaption capability when used with additional unlabeled data.