Delving Deep into Pixel Alignment Feature for Accurate Multi-view Human Mesh Recovery
This work addresses accurate 3D human pose and shape estimation from multiple camera views, which is incremental as it builds on existing regression-based methods with specific enhancements.
The paper tackles multi-view human mesh recovery by introducing Pixel-aligned Feedback Fusion (PaFF), an iterative regression framework that alternates feature extraction and fusion, achieving 33.02 MPJPE on the Human3.6M dataset with a 29% improvement over prior methods.
Regression-based methods have shown high efficiency and effectiveness for multi-view human mesh recovery. The key components of a typical regressor lie in the feature extraction of input views and the fusion of multi-view features. In this paper, we present Pixel-aligned Feedback Fusion (PaFF) for accurate yet efficient human mesh recovery from multi-view images. PaFF is an iterative regression framework that performs feature extraction and fusion alternately. At each iteration, PaFF extracts pixel-aligned feedback features from each input view according to the reprojection of the current estimation and fuses them together with respect to each vertex of the downsampled mesh. In this way, our regressor can not only perceive the misalignment status of each view from the feedback features but also correct the mesh parameters more effectively based on the feature fusion on mesh vertices. Additionally, our regressor disentangles the global orientation and translation of the body mesh from the estimation of mesh parameters such that the camera parameters of input views can be better utilized in the regression process. The efficacy of our method is validated in the Human3.6M dataset via comprehensive ablation experiments, where PaFF achieves 33.02 MPJPE and brings significant improvements over the previous best solutions by more than 29%. The project page with code and video results can be found at https://kairobo.github.io/PaFF/.