Cross-Modal Diffusion for Biomechanical Dynamical Systems Through Local Manifold Alignment
This work addresses the problem of generating accurate biomechanical motions for applications like robotics or healthcare, but it is incremental as it builds on existing diffusion frameworks with a novel alignment approach.
The paper tackled cross-modal biomechanical motion generation by aligning latent representations of joint angles and ground reaction forces using a local manifold alignment strategy, resulting in improved generation fidelity and better representations in experiments on multimodal human biomechanics data.
We present a mutually aligned diffusion framework for cross-modal biomechanical motion generation, guided by a dynamical systems perspective. By treating each modality, e.g., observed joint angles ($X$) and ground reaction forces ($Y$), as complementary observations of a shared underlying locomotor dynamical system, our method aligns latent representations at each diffusion step, so that one modality can help denoise and disambiguate the other. Our alignment approach is motivated by the fact that local time windows of $X$ and $Y$ represent the same phase of an underlying dynamical system, thereby benefiting from a shared latent manifold. We introduce a simple local latent manifold alignment (LLMA) strategy that incorporates first-order and second-order alignment within the latent space for robust cross-modal biomechanical generation without bells and whistles. Through experiments on multimodal human biomechanics data, we show that aligning local latent dynamics across modalities improves generation fidelity and yields better representations.