Topological Eulerian Synthesis of Slow Motion Periodic Videos
This work solves the problem of creating clean, single-cycle motion videos for applications in video processing and analysis, though it is incremental as it builds on existing Eulerian and topological approaches.
The paper tackles the problem of synthesizing a single, detailed cycle of motion from videos with repetitive motion by reordering frames, addressing challenges like noise, camera drift, and occlusions. It introduces a tracking-free Eulerian method using topological and spectral analysis, showing robustness to camera shake, noise, and occlusions in quantitative and qualitative results.
We consider the problem of taking a video that is comprised of multiple periods of repetitive motion, and reordering the frames of the video into a single period, producing a detailed, single cycle video of motion. This problem is challenging, as such videos often contain noise, drift due to camera motion and from cycle to cycle, and irrelevant background motion/occlusions, and these factors can confound the relevant periodic motion we seek in the video. To address these issues in a simple and efficient manner, we introduce a tracking free Eulerian approach for synthesizing a single cycle of motion. Our approach is geometric: we treat each frame as a point in high-dimensional Euclidean space, and analyze the sliding window embedding formed by this sequence of points, which yields samples along a topological loop regardless of the type of periodic motion. We combine tools from topological data analysis and spectral geometric analysis to estimate the phase of each window, and we exploit the sliding window structure to robustly reorder frames. We show quantitative results that highlight the robustness of our technique to camera shake, noise, and occlusions, and qualitative results of single-cycle motion synthesis across a variety of scenarios.