A Primer on Motion Capture with Deep Learning: Principles, Pitfalls and Perspectives
This is an incremental primer that provides an overview for researchers in neuroscience and biology on applying deep learning to motion capture.
The paper reviews the field of motion capture using deep learning, addressing the hard computational problem of non-invasive behavioral measurement from video, and highlights the principles, potential, and pitfalls of these algorithms for experimentalists.
Extracting behavioral measurements non-invasively from video is stymied by the fact that it is a hard computational problem. Recent advances in deep learning have tremendously advanced predicting posture from videos directly, which quickly impacted neuroscience and biology more broadly. In this primer we review the budding field of motion capture with deep learning. In particular, we will discuss the principles of those novel algorithms, highlight their potential as well as pitfalls for experimentalists, and provide a glimpse into the future.