Prime and Reach: Synthesising Body Motion for Gaze-Primed Object Reach
This addresses the problem of synthesizing natural human motion for robotics or animation, but it is incremental as it builds on existing datasets and methods.
The paper tackled generating realistic human motion for gaze-primed object reaching by curating 23.7K sequences from five datasets and fine-tuning a diffusion model, achieving 60% prime success and 89% reach success on HD-EPIC.
Human motion generation is a challenging task that aims to create realistic motion imitating natural human behaviour. We focus on the well-studied behaviour of priming an object/location for pick up or put down -- that is, the spotting of an object/location from a distance, known as gaze priming, followed by the motion of approaching and reaching the target location. To that end, we curate, for the first time, 23.7K gaze-primed human motion sequences for reaching target object locations from five publicly available datasets, i.e., HD-EPIC, MoGaze, HOT3D, ADT, and GIMO. We pre-train a text-conditioned diffusion-based motion generation model, then fine-tune it conditioned on goal pose or location, on our curated sequences. Importantly, we evaluate the ability of the generated motion to imitate natural human movement through several metrics, including the 'Reach Success' and a newly introduced 'Prime Success' metric. On the largest dataset, HD-EPIC, our model achieves 60% prime success and 89% reach success when conditioned on the goal object location.