EgoCap: Egocentric Marker-less Motion Capture with Two Fisheye Cameras (Extended Abstract)
This addresses the discomfort and limited recording volume of traditional motion capture systems for users in various environments.
The paper tackles the problem of restricted and uncomfortable motion capture by proposing a real-time, marker-less, egocentric method using a stereo pair of fisheye cameras attached to a helmet or VR headset, which captures full-body motion in general indoor, outdoor, and crowded scenes.
Marker-based and marker-less optical skeletal motion-capture methods use an outside-in arrangement of cameras placed around a scene, with viewpoints converging on the center. They often create discomfort by possibly needed marker suits, and their recording volume is severely restricted and often constrained to indoor scenes with controlled backgrounds. We therefore propose a new method for real-time, marker-less and egocentric motion capture which estimates the full-body skeleton pose from a lightweight stereo pair of fisheye cameras that are attached to a helmet or virtual-reality headset. It combines the strength of a new generative pose estimation framework for fisheye views with a ConvNet-based body-part detector trained on a new automatically annotated and augmented dataset. Our inside-in method captures full-body motion in general indoor and outdoor scenes, and also crowded scenes.