Object-Centric Multi-Task Learning for Human Instances
This addresses the need for efficient multi-task learning in visual recognition for human instances, though it is incremental as it builds on existing object-centric and transformer-based methods.
The paper tackles the problem of multi-task learning for human detection, segmentation, and pose estimation by proposing a compact network architecture with a human-centric query design, achieving comparable accuracy to state-of-the-art task-specific models while reducing computational costs.
Human is one of the most essential classes in visual recognition tasks such as detection, segmentation, and pose estimation. Although much effort has been put into individual tasks, multi-task learning for these three tasks has been rarely studied. In this paper, we explore a compact multi-task network architecture that maximally shares the parameters of the multiple tasks via object-centric learning. To this end, we propose a novel query design to encode the human instance information effectively, called human-centric query (HCQ). HCQ enables for the query to learn explicit and structural information of human as well such as keypoints. Besides, we utilize HCQ in prediction heads of the target tasks directly and also interweave HCQ with the deformable attention in Transformer decoders to exploit a well-learned object-centric representation. Experimental results show that the proposed multi-task network achieves comparable accuracy to state-of-the-art task-specific models in human detection, segmentation, and pose estimation task, while it consumes less computational costs.