TranSplat: Lighting-Consistent Cross-Scene Object Transfer with 3D Gaussian Splatting
This addresses the challenge of lighting-consistent object insertion in 3D scenes for applications like augmented reality or visual effects, though it is incremental as it builds on the Gaussian Splatting framework.
The paper tackles the problem of realistic cross-scene object transfer by enabling precise 3D object extraction and faithful relighting without explicit material estimation, resulting in excellent performance compared to baseline methods.
We present TranSplat, a 3D scene rendering algorithm that enables realistic cross-scene object transfer (from a source to a target scene) based on the Gaussian Splatting framework. Our approach addresses two critical challenges: (1) precise 3D object extraction from the source scene, and (2) faithful relighting of the transferred object in the target scene without explicit material property estimation. TranSplat fits a splatting model to the source scene, using 2D object masks to drive fine-grained 3D segmentation. Following user-guided insertion of the object into the target scene, along with automatic refinement of position and orientation, TranSplat derives per-Gaussian radiance transfer functions via spherical harmonic analysis to adapt the object's appearance to match the target scene's lighting environment. This relighting strategy does not require explicitly estimating physical scene properties such as BRDFs. Evaluated on several synthetic and real-world scenes and objects, TranSplat yields excellent 3D object extractions and relighting performance compared to recent baseline methods and visually convincing cross-scene object transfers. We conclude by discussing the limitations of the approach.