Animated 3DGS Avatars in Diverse Scenes with Consistent Lighting and Shadows
This addresses the challenge of realistic avatar integration in 3D scenes for applications in virtual reality, gaming, and film, though it is incremental as it builds on existing 3DGS and shadow mapping techniques.
The paper tackles the problem of achieving consistent lighting and shadows for animated 3D Gaussian Splatting avatars in diverse scenes, resulting in a method that enables real-time volumetric shadow computation and environment relighting without meshing, as demonstrated with avatars from datasets like AvatarX and ActorsHQ in scenes such as ScanNet++ and DL3DV.
We present a method for consistent lighting and shadows when animated 3D Gaussian Splatting (3DGS) avatars interact with 3DGS scenes or with dynamic objects inserted into otherwise static scenes. Our key contribution is Deep Gaussian Shadow Maps (DGSM), a modern analogue of the classical shadow mapping algorithm tailored to the volumetric 3DGS representation. Building on the classic deep shadow mapping idea, we show that 3DGS admits closed form light accumulation along light rays, enabling volumetric shadow computation without meshing. For each estimated light, we tabulate transmittance over concentric radial shells and store them in octahedral atlases, which modern GPUs can sample in real time per query to attenuate affected scene Gaussians and thus cast and receive shadows consistently. To relight moving avatars, we approximate the local environment illumination with HDRI probes represented in a spherical harmonic (SH) basis and apply a fast per Gaussian radiance transfer, avoiding explicit BRDF estimation or offline optimization. We demonstrate environment consistent lighting for avatars from AvatarX and ActorsHQ, composited into ScanNet++, DL3DV, and SuperSplat scenes, and show interactions with inserted objects. Across single and multi avatar settings, DGSM and SH relighting operate fully in the volumetric 3DGS representation, yielding coherent shadows and relighting while avoiding meshing.