CIRCLE: Capture In Rich Contextual Environments
This addresses the lack of rich contextual 3D human motion datasets for creating realistic generative human motion models, which is incremental as it builds on existing motion capture and virtual reality technologies.
The paper tackles the problem of synthesizing 3D human motion in contextual environments by proposing a novel motion acquisition system that captures high-quality motion in virtual scenes, resulting in a dataset of 10 hours of full-body reaching motion from 5 subjects across nine scenes and a model that generates motion conditioned on scene information.
Synthesizing 3D human motion in a contextual, ecological environment is important for simulating realistic activities people perform in the real world. However, conventional optics-based motion capture systems are not suited for simultaneously capturing human movements and complex scenes. The lack of rich contextual 3D human motion datasets presents a roadblock to creating high-quality generative human motion models. We propose a novel motion acquisition system in which the actor perceives and operates in a highly contextual virtual world while being motion captured in the real world. Our system enables rapid collection of high-quality human motion in highly diverse scenes, without the concern of occlusion or the need for physical scene construction in the real world. We present CIRCLE, a dataset containing 10 hours of full-body reaching motion from 5 subjects across nine scenes, paired with ego-centric information of the environment represented in various forms, such as RGBD videos. We use this dataset to train a model that generates human motion conditioned on scene information. Leveraging our dataset, the model learns to use ego-centric scene information to achieve nontrivial reaching tasks in the context of complex 3D scenes. To download the data please visit https://stanford-tml.github.io/circle_dataset/.