CoopDiff: Anticipating 3D Human-object Interactions via Contact-consistent Decoupled Diffusion
This work addresses the problem of predicting future human-object interactions in 3D for applications like robotics and VR, but it is incremental as it builds on existing diffusion models with a decoupled approach.
The paper tackles 3D human-object interaction anticipation by proposing CoopDiff, a contact-consistent decoupled diffusion framework that uses separate branches for human and object motion modeling, bridged by shared contact points, and it outperforms state-of-the-art methods on BEHAVE and Human-object Interaction datasets.
3D human-object interaction (HOI) anticipation aims to predict the future motion of humans and their manipulated objects, conditioned on the historical context. Generally, the articulated humans and rigid objects exhibit different motion patterns, due to their distinct intrinsic physical properties. However, this distinction is ignored by most of the existing works, which intend to capture the dynamics of both humans and objects within a single prediction model. In this work, we propose a novel contact-consistent decoupled diffusion framework CoopDiff, which employs two distinct branches to decouple human and object motion modeling, with the human-object contact points as shared anchors to bridge the motion generation across branches. The human dynamics branch is aimed to predict highly structured human motion, while the object dynamics branch focuses on the object motion with rigid translations and rotations. These two branches are bridged by a series of shared contact points with consistency constraint for coherent human-object motion prediction. To further enhance human-object consistency and prediction reliability, we propose a human-driven interaction module to guide object motion modeling. Extensive experiments on the BEHAVE and Human-object Interaction datasets demonstrate that our CoopDiff outperforms state-of-the-art methods.