Learnable Triangulation of Human Pose
This work addresses the problem of accurate 3D human pose estimation from multiple camera views for applications in computer vision, representing a strong specific gain rather than a foundational breakthrough.
The paper tackles multi-view 3D human pose estimation by introducing two learnable triangulation methods that combine 2D views into 3D poses, resulting in a significant improvement in state-of-the-art performance on the Human3.6M dataset.
We present two novel solutions for multi-view 3D human pose estimation based on new learnable triangulation methods that combine 3D information from multiple 2D views. The first (baseline) solution is a basic differentiable algebraic triangulation with an addition of confidence weights estimated from the input images. The second solution is based on a novel method of volumetric aggregation from intermediate 2D backbone feature maps. The aggregated volume is then refined via 3D convolutions that produce final 3D joint heatmaps and allow modelling a human pose prior. Crucially, both approaches are end-to-end differentiable, which allows us to directly optimize the target metric. We demonstrate transferability of the solutions across datasets and considerably improve the multi-view state of the art on the Human3.6M dataset. Video demonstration, annotations and additional materials will be posted on our project page (https://saic-violet.github.io/learnable-triangulation).