Learning Spatial Context with Graph Neural Network for Multi-Person Pose Grouping
This work addresses the grouping stage in bottom-up multi-person pose estimation for computer vision applications, but it is incremental as it builds on existing methods by adding spatial information.
The paper tackled the problem of grouping detected keypoints into person instances in multi-person pose estimation by incorporating spatial context, and the result showed that their method outperformed existing appearance-only grouping frameworks on two benchmark datasets.
Bottom-up approaches for image-based multi-person pose estimation consist of two stages: (1) keypoint detection and (2) grouping of the detected keypoints to form person instances. Current grouping approaches rely on learned embedding from only visual features that completely ignore the spatial configuration of human poses. In this work, we formulate the grouping task as a graph partitioning problem, where we learn the affinity matrix with a Graph Neural Network (GNN). More specifically, we design a Geometry-aware Association GNN that utilizes spatial information of the keypoints and learns local affinity from the global context. The learned geometry-based affinity is further fused with appearance-based affinity to achieve robust keypoint association. Spectral clustering is used to partition the graph for the formation of the pose instances. Experimental results on two benchmark datasets show that our proposed method outperforms existing appearance-only grouping frameworks, which shows the effectiveness of utilizing spatial context for robust grouping. Source code is available at: https://github.com/jiahaoLjh/PoseGrouping.