VirtualFencer: Generating Fencing Bouts based on Strategies Extracted from In-the-Wild Videos
This addresses the challenge of modeling diverse and strategic two-player motions in sports like fencing, though it appears incremental as it applies data-driven modeling to a specific domain.
The paper tackled the problem of generating realistic fencing bouts by extracting 3D motion and strategy from in-the-wild videos without supervision, resulting in a system that can fence against itself, real fencer motions, and interactively against a professional fencer.
Fencing is a sport where athletes engage in diverse yet strategically logical motions. While most motions fall into a few high-level actions (e.g. step, lunge, parry), the execution can vary widely-fast vs. slow, large vs. small, offensive vs. defensive. Moreover, a fencer's actions are informed by a strategy that often comes in response to the opponent's behavior. This combination of motion diversity with underlying two-player strategy motivates the application of data-driven modeling to fencing. We present VirtualFencer, a system capable of extracting 3D fencing motion and strategy from in-the-wild video without supervision, and then using that extracted knowledge to generate realistic fencing behavior. We demonstrate the versatile capabilities of our system by having it (i) fence against itself (self-play), (ii) fence against a real fencer's motion from online video, and (iii) fence interactively against a professional fencer.