Synthesizing Long-Term 3D Human Motion and Interaction in 3D Scenes
This work is significant for graphics applications and human activity understanding by enabling more realistic and interactive 3D human motion synthesis in complex scenes, which is an incremental improvement over existing methods.
This paper addresses the problem of synthesizing long-term 3D human motion within 3D scenes, specifically focusing on human-scene interactions and affordance. The authors propose a hierarchical generative framework that incorporates geometry constraints between the human mesh and scene point clouds, leading to significant improvements in generating natural and physically plausible human motion compared to prior methods.
Synthesizing 3D human motion plays an important role in many graphics applications as well as understanding human activity. While many efforts have been made on generating realistic and natural human motion, most approaches neglect the importance of modeling human-scene interactions and affordance. On the other hand, affordance reasoning (e.g., standing on the floor or sitting on the chair) has mainly been studied with static human pose and gestures, and it has rarely been addressed with human motion. In this paper, we propose to bridge human motion synthesis and scene affordance reasoning. We present a hierarchical generative framework to synthesize long-term 3D human motion conditioning on the 3D scene structure. Building on this framework, we further enforce multiple geometry constraints between the human mesh and scene point clouds via optimization to improve realistic synthesis. Our experiments show significant improvements over previous approaches on generating natural and physically plausible human motion in a scene.