LATTE-MV: Learning to Anticipate Table Tennis Hits from Monocular Videos
This work addresses the challenge of real-time anticipation in fast-paced sports like table tennis, though it is incremental as it builds on existing systems by adding anticipation capabilities.
The paper tackles the problem of anticipating opponent actions in table tennis from monocular videos, resulting in a policy that improves the ball return rate from 49.9% to 59.0% in simulation.
Physical agility is a necessary skill in competitive table tennis, but by no means sufficient. Champions excel in this fast-paced and highly dynamic environment by anticipating their opponent's intent - buying themselves the necessary time to react. In this work, we take one step towards designing such an anticipatory agent. Previous works have developed systems capable of real-time table tennis gameplay, though they often do not leverage anticipation. Among the works that forecast opponent actions, their approaches are limited by dataset size and variety. Our paper contributes (1) a scalable system for reconstructing monocular video of table tennis matches in 3D and (2) an uncertainty-aware controller that anticipates opponent actions. We demonstrate in simulation that our policy improves the ball return rate against high-speed hits from 49.9% to 59.0% as compared to a baseline non-anticipatory policy.