Intrinsic decomposition and editing of 3D Gaussian splats
This work addresses the problem of photorealistic editing of 3D scenes represented by Gaussian splatting, enabling intuitive texture editing with consistent lighting across views.
The paper extends intrinsic image decomposition to 3D Gaussian splatting radiance fields, enabling editing of object colors and textures without altering lighting. It proposes a method to decompose radiance fields into albedo and shading layers, guided by data-driven predictions, and demonstrates editing by modifying albedo in a single image.
Intrinsic decomposition which expresses image colors as the product of diffuse albedo and shading, possibly augmented with view-dependent residuals has a long history in image editing as it enables the modification of object colors and textures without altering lighting. We extend intrinsic decomposition to radiance fields represented with Gaussian splatting by proposing solutions to three key aspects of such decomposition. First, we describe how to model the intrinsic decomposition as independent sets of Gaussian primitives, which allows each set to adapt to the characteristics of the layer it represents. Second, we present an optimization procedure guided by data-driven predictions to disentangle multi-view photographs of a scene into the aforementioned intrinsic sets. Finally, we provide an editing workflow where users modify the texture of planar surfaces simply by modifying the albedo of that surface in one image. Capturing this edit within the intrinsic radiance field allows re-rendering of the edited scene with plausible lighting under arbitrary viewpoints.