TRB: A Novel Triplet Representation for Understanding 2D Human Body
This addresses the need for a compact ody representation in computer vision, enabling applications like human shape editing and conditional image generation, but it appears incremental as it builds on existing pose datasets and methods.
The paper tackles the problem of efficiently representing both human pose and shape in 2D images by proposing the Triplet Representation for Body (TRB), which combines skeleton and contour keypoints, and introduces a two-branch network (TRB-net) with novel techniques to solve TRB estimation, demonstrating superiority over leading approaches on constructed datasets.
Human pose and shape are two important components of 2D human body. However, how to efficiently represent both of them in images is still an open question. In this paper, we propose the Triplet Representation for Body (TRB) -- a compact 2D human body representation, with skeleton keypoints capturing human pose information and contour keypoints containing human shape information. TRB not only preserves the flexibility of skeleton keypoint representation, but also contains rich pose and human shape information. Therefore, it promises broader application areas, such as human shape editing and conditional image generation. We further introduce the challenging problem of TRB estimation, where joint learning of human pose and shape is required. We construct several large-scale TRB estimation datasets, based on popular 2D pose datasets: LSP, MPII, COCO. To effectively solve TRB estimation, we propose a two-branch network (TRB-net) with three novel techniques, namely X-structure (Xs), Directional Convolution (DC) and Pairwise Mapping (PM), to enforce multi-level message passing for joint feature learning. We evaluate our proposed TRB-net and several leading approaches on our proposed TRB datasets, and demonstrate the superiority of our method through extensive evaluations.