LieHMR: Autoregressive Human Mesh Recovery with $SO(3)$ Diffusion
This work addresses the inherent ambiguity in 3D human pose estimation from 2D images for applications in computer vision and graphics, representing an incremental improvement over prior probabilistic approaches.
The paper tackles the problem of Human Mesh Recovery from a single RGB image by proposing a novel SO(3) diffusion model to generate accurate pose probability distributions, achieving competitive accuracy and diversity compared to existing probabilistic and deterministic methods.
We tackle the problem of Human Mesh Recovery (HMR) from a single RGB image, formulating it as an image-conditioned human pose and shape generation. While recovering 3D human pose from 2D observations is inherently ambiguous, most existing approaches have regressed a single deterministic output. Probabilistic methods attempt to address this by generating multiple plausible outputs to model the ambiguity. However, these methods often exhibit a trade-off between accuracy and sample diversity, and their single predictions are not competitive with state-of-the-art deterministic models. To overcome these limitations, we propose a novel approach that models well-aligned distribution to 2D observations. In particular, we introduce $SO(3)$ diffusion model, which generates the distribution of pose parameters represented as 3D rotations unconditional and conditional to image observations via conditioning dropout. Our model learns the hierarchical structure of human body joints using the transformer. Instead of using transformer as a denoising model, the time-independent transformer extracts latent vectors for the joints and a small MLP-based denoising model learns the per-joint distribution conditioned on the latent vector. We experimentally demonstrate and analyze that our model predicts accurate pose probability distribution effectively.