H-MoRe: Learning Human-centric Motion Representation for Action Analysis
This work addresses the need for accurate human motion analysis in action-related applications, offering a novel self-supervised method that outperforms previous supervised approaches.
The paper tackles the problem of learning precise human-centric motion representation by proposing H-MoRe, a self-supervised pipeline that dynamically filters background movement and incorporates human pose and body shape information, resulting in substantial improvements such as +16.01% in gait recognition, +8.92% in action recognition, and -67.07% in video generation FVD.
In this paper, we propose H-MoRe, a novel pipeline for learning precise human-centric motion representation. Our approach dynamically preserves relevant human motion while filtering out background movement. Notably, unlike previous methods relying on fully supervised learning from synthetic data, H-MoRe learns directly from real-world scenarios in a self-supervised manner, incorporating both human pose and body shape information. Inspired by kinematics, H-MoRe represents absolute and relative movements of each body point in a matrix format that captures nuanced motion details, termed world-local flows. H-MoRe offers refined insights into human motion, which can be integrated seamlessly into various action-related applications. Experimental results demonstrate that H-MoRe brings substantial improvements across various downstream tasks, including gait recognition(CL@R1: +16.01%), action recognition(Acc@1: +8.92%), and video generation(FVD: -67.07%). Additionally, H-MoRe exhibits high inference efficiency (34 fps), making it suitable for most real-time scenarios. Models and code will be released upon publication.