Data exploitation: multi-task learning of object detection and semantic segmentation on partially annotated data
This work addresses data scarcity in computer vision by enabling efficient use of partially annotated datasets, which is incremental but practical for reducing annotation costs.
The paper tackles the problem of learning object detection and semantic segmentation from partially annotated data, where each image is labeled for only one task, and shows that multi-task learning with knowledge distillation outperforms single-task learning and even fully supervised scenarios.
Multi-task partially annotated data where each data point is annotated for only a single task are potentially helpful for data scarcity if a network can leverage the inter-task relationship. In this paper, we study the joint learning of object detection and semantic segmentation, the two most popular vision problems, from multi-task data with partial annotations. Extensive experiments are performed to evaluate each task performance and explore their complementarity when a multi-task network cannot optimize both tasks simultaneously. We propose employing knowledge distillation to leverage joint-task optimization. The experimental results show favorable results for multi-task learning and knowledge distillation over single-task learning and even full supervision scenario. All code and data splits are available at https://github.com/lhoangan/multas