InfoGaussian: Structure-Aware Dynamic Gaussians through Lightweight Information Shaping
This addresses the challenge of animating objects in 3D Gaussian scenes for computer graphics applications, representing an incremental improvement through a novel method for a known bottleneck.
The paper tackles the problem of controlling 3D Gaussian scene representations for animation by developing a mutual information shaping technique that enforces coordinated movement among correlated Gaussians, achieving significant performance improvement in consistent movements and 3D object segmentation with low computation and memory requirements.
3D Gaussians, as a low-level scene representation, typically involve thousands to millions of Gaussians. This makes it difficult to control the scene in ways that reflect the underlying dynamic structure, where the number of independent entities is typically much smaller. In particular, it can be challenging to animate and move objects in the scene, which requires coordination among many Gaussians. To address this issue, we develop a mutual information shaping technique that enforces movement resonance between correlated Gaussians in a motion network. Such correlations can be learned from putative 2D object masks in different views. By approximating the mutual information with the Jacobians of the motions, our method ensures consistent movements of the Gaussians composing different objects under various perturbations. In particular, we develop an efficient contrastive training pipeline with lightweight optimization to shape the motion network, avoiding the need for re-shaping throughout the motion sequence. Notably, our training only touches a small fraction of all Gaussians in the scene yet attains the desired compositional behavior according to the underlying dynamic structure. The proposed technique is evaluated on challenging scenes and demonstrates significant performance improvement in promoting consistent movements and 3D object segmentation while inducing low computation and memory requirements.