Dynamic Avatar-Scene Rendering from Human-centric Context
This addresses the challenge of creating realistic 4D reconstructions of humans in scenes for applications like VR/AR, though it appears incremental relative to prior neural rendering approaches.
The paper tackles the problem of reconstructing dynamic humans interacting with real-world environments from monocular videos, proposing a Separate-then-Map strategy that significantly outperforms existing methods in visual quality and rendering accuracy, especially at human-scene boundaries.
Reconstructing dynamic humans interacting with real-world environments from monocular videos is an important and challenging task. Despite considerable progress in 4D neural rendering, existing approaches either model dynamic scenes holistically or model scenes and backgrounds separately aim to introduce parametric human priors. However, these approaches either neglect distinct motion characteristics of various components in scene especially human, leading to incomplete reconstructions, or ignore the information exchange between the separately modeled components, resulting in spatial inconsistencies and visual artifacts at human-scene boundaries. To address this, we propose {\bf Separate-then-Map} (StM) strategy that introduces a dedicated information mapping mechanism to bridge separately defined and optimized models. Our method employs a shared transformation function for each Gaussian attribute to unify separately modeled components, enhancing computational efficiency by avoiding exhaustive pairwise interactions while ensuring spatial and visual coherence between humans and their surroundings. Extensive experiments on monocular video datasets demonstrate that StM significantly outperforms existing state-of-the-art methods in both visual quality and rendering accuracy, particularly at challenging human-scene interaction boundaries.