UnrealPose: Leveraging Game Engine Kinematics for Large-Scale Synthetic Human Pose Data
This provides a scalable solution for generating synthetic human pose data to address data scarcity in computer vision, though it is incremental as it builds on existing game engine and rendering techniques.
The authors tackled the problem of expensive and limited 3D human pose data by introducing UnrealPose-Gen, a pipeline using Unreal Engine 5 to generate synthetic data, resulting in the UnrealPose-1M dataset with approximately one million frames and validation on four pose-related tasks.
Diverse, accurately labeled 3D human pose data is expensive and studio-bound, while in-the-wild datasets lack known ground truth. We introduce UnrealPose-Gen, an Unreal Engine 5 pipeline built on Movie Render Queue for high-quality offline rendering. Our generated frames include: (i) 3D joints in world and camera coordinates, (ii) 2D projections and COCO-style keypoints with occlusion and joint-visibility flags, (iii) person bounding boxes, and (iv) camera intrinsics and extrinsics. We use UnrealPose-Gen to present UnrealPose-1M, an approximately one million frame corpus comprising eight sequences: five scripted "coherent" sequences spanning five scenes, approximately 40 actions, and five subjects; and three randomized sequences across three scenes, approximately 100 actions, and five subjects, all captured from diverse camera trajectories for broad viewpoint coverage. As a fidelity check, we report real-to-synthetic results on four tasks: image-to-3D pose, 2D keypoint detection, 2D-to-3D lifting, and person detection/segmentation. Though time and resources constrain us from an unlimited dataset, we release the UnrealPose-1M dataset, as well as the UnrealPose-Gen pipeline to support third-party generation of human pose data.