Gaussian See, Gaussian Do: Semantic 3D Motion Transfer from Multiview Video
This work addresses the challenge of transferring motion between objects in 3D without rigging, which is incremental as it builds on implicit motion transfer techniques.
The paper tackles the problem of semantic 3D motion transfer from multiview video by enabling rig-free, cross-category motion transfer with semantic correspondence, resulting in superior motion fidelity and structural consistency compared to adapted baselines.
We present Gaussian See, Gaussian Do, a novel approach for semantic 3D motion transfer from multiview video. Our method enables rig-free, cross-category motion transfer between objects with semantically meaningful correspondence. Building on implicit motion transfer techniques, we extract motion embeddings from source videos via condition inversion, apply them to rendered frames of static target shapes, and use the resulting videos to supervise dynamic 3D Gaussian Splatting reconstruction. Our approach introduces an anchor-based view-aware motion embedding mechanism, ensuring cross-view consistency and accelerating convergence, along with a robust 4D reconstruction pipeline that consolidates noisy supervision videos. We establish the first benchmark for semantic 3D motion transfer and demonstrate superior motion fidelity and structural consistency compared to adapted baselines. Code and data for this paper available at https://gsgd-motiontransfer.github.io/