Rendering Multi-Human and Multi-Object with 3D Gaussian Splatting
This work addresses a critical challenge for creating high-fidelity digital twins in robotics and VR/AR, but it is incremental as it builds upon existing 3D Gaussian Splatting methods.
The paper tackles the problem of reconstructing dynamic scenes with multiple interacting humans and objects from sparse-view inputs, termed Multi-Human Multi-Object (MHMO) rendering, and proposes MM-GS, a hierarchical framework based on 3D Gaussian Splatting that significantly outperforms strong baselines, achieving state-of-the-art results with high-fidelity details and plausible inter-instance contacts.
Reconstructing dynamic scenes with multiple interacting humans and objects from sparse-view inputs is a critical yet challenging task, essential for creating high-fidelity digital twins for robotics and VR/AR. This problem, which we term Multi-Human Multi-Object (MHMO) rendering, presents two significant obstacles: achieving view-consistent representations for individual instances under severe mutual occlusion, and explicitly modeling the complex and combinatorial dependencies that arise from their interactions. To overcome these challenges, we propose MM-GS, a novel hierarchical framework built upon 3D Gaussian Splatting. Our method first employs a Per-Instance Multi-View Fusion module to establish a robust and consistent representation for each instance by aggregating visual information across all available views. Subsequently, a Scene-Level Instance Interaction module operates on a global scene graph to reason about relationships between all participants, refining their attributes to capture subtle interaction effects. Extensive experiments on challenging datasets demonstrate that our method significantly outperforms strong baselines, producing state-of-the-art results with high-fidelity details and plausible inter-instance contacts.