Gaussian Shadow Casting for Neural Characters
This addresses the problem of realistic shadow generation for neural characters, enabling more applicable posing in novel environments, though it is an incremental improvement over existing methods like Gaussian splatting.
The paper tackles the problem of neural character models lacking explicit shadows and shading, which causes artifacts in novel views, poses, and relighting. It proposes a Gaussian density proxy for shadows, resulting in improved reconstructions with better separation of albedo, shading, and shadows in challenging outdoor scenes, and fewer shadow artifacts in novel poses.
Neural character models can now reconstruct detailed geometry and texture from video, but they lack explicit shadows and shading, leading to artifacts when generating novel views and poses or during relighting. It is particularly difficult to include shadows as they are a global effect and the required casting of secondary rays is costly. We propose a new shadow model using a Gaussian density proxy that replaces sampling with a simple analytic formula. It supports dynamic motion and is tailored for shadow computation, thereby avoiding the affine projection approximation and sorting required by the closely related Gaussian splatting. Combined with a deferred neural rendering model, our Gaussian shadows enable Lambertian shading and shadow casting with minimal overhead. We demonstrate improved reconstructions, with better separation of albedo, shading, and shadows in challenging outdoor scenes with direct sun light and hard shadows. Our method is able to optimize the light direction without any input from the user. As a result, novel poses have fewer shadow artifacts and relighting in novel scenes is more realistic compared to the state-of-the-art methods, providing new ways to pose neural characters in novel environments, increasing their applicability.