Discriminative-Generative Synergy for Occlusion Robust 3D Human Mesh Recovery
For researchers in 3D human pose and shape estimation, this work addresses the occlusion robustness problem by synergizing discriminative and generative approaches, though the gains are incremental over existing methods.
The paper tackles 3D human mesh recovery from monocular RGB images under occlusions by integrating a ViT-based discriminative pathway with a diffusion-based generative pathway. The method achieves superior performance on standard benchmarks and shows strong robustness in complex real-world scenarios.
3D human mesh recovery from monocular RGB images aims to estimate anatomically plausible 3D human models for downstream applications, but remains challenging under partial or severe occlusions. Regression-based methods are efficient yet often produce implausible or inaccurate results in unconstrained scenarios, while diffusion-based methods provide strong generative priors for occluded regions but may weaken fidelity to rare poses due to over-reliance on generation. To address these limitations, we propose a brain-inspired synergistic framework that integrates the discriminative power of vision transformers with the generative capability of conditional diffusion models. Specifically, the ViT-based pathway extracts deterministic visual cues from visible regions, while the diffusion-based pathway synthesizes structurally coherent human body representations. To effectively bridge the two pathways, we design a diverse-consistent feature learning module to align discriminative features with generative priors, and a cross-attention multi-level fusion mechanism to enable bidirectional interaction across semantic levels. Experiments on standard benchmarks demonstrate that our method achieves superior performance on key metrics and shows strong robustness in complex real-world scenarios.