Pose Priors from Language Models
This work addresses the challenge of incorporating language-based physical interaction descriptions into 3D human pose estimation, providing a scalable solution that reduces reliance on costly manual annotations or motion capture data.
The paper tackles the problem of 3D human pose estimation by leveraging large multimodal models (LMMs) as priors to reconstruct contact poses, offering a scalable alternative to methods relying on human annotations or motion capture data. The method produces compelling reconstructions for two-person interactions and self-contact scenarios, accurately capturing interaction semantics.
Language is often used to describe physical interaction, yet most 3D human pose estimation methods overlook this rich source of information. We bridge this gap by leveraging large multimodal models (LMMs) as priors for reconstructing contact poses, offering a scalable alternative to traditional methods that rely on human annotations or motion capture data. Our approach extracts contact-relevant descriptors from an LMM and translates them into tractable losses to constrain 3D human pose optimization. Despite its simplicity, our method produces compelling reconstructions for both two-person interactions and self-contact scenarios, accurately capturing the semantics of physical and social interactions. Our results demonstrate that LMMs can serve as powerful tools for contact prediction and pose estimation, offering an alternative to costly manual human annotations or motion capture data. Our code is publicly available at https://prosepose.github.io.