Mojito: LLM-Aided Motion Instructor with Jitter-Reduced Inertial Tokens
This addresses motion analysis for applications requiring real-time, privacy-conscious sensing, but appears incremental as it combines existing technologies (IMUs and LLMs) without claiming major breakthroughs.
The paper tackles the problem of capturing and analyzing human motion using inertial measurement units (IMUs), which face issues like noise and drift, by introducing Mojito, an agent that integrates IMUs with large language models for interactive motion capture and analysis, though no concrete results or numbers are provided.
Human bodily movements convey critical insights into action intentions and cognitive processes, yet existing multimodal systems primarily focused on understanding human motion via language, vision, and audio, which struggle to capture the dynamic forces and torques inherent in 3D motion. Inertial measurement units (IMUs) present a promising alternative, offering lightweight, wearable, and privacy-conscious motion sensing. However, processing of streaming IMU data faces challenges such as wireless transmission instability, sensor noise, and drift, limiting their utility for long-term real-time motion capture (MoCap), and more importantly, online motion analysis. To address these challenges, we introduce Mojito, an intelligent motion agent that integrates inertial sensing with large language models (LLMs) for interactive motion capture and behavioral analysis.