Capture and Interact: Rapid 3D Object Acquisition and Rendering with Gaussian Splatting in Unity
This enables rapid 3D object acquisition for applications like augmented reality and digital twins, though it is incremental as it builds on existing 3D Gaussian Splatting methods.
The authors tackled the challenge of real-time 3D object capture and rendering by developing an end-to-end pipeline using 3D Gaussian Splatting, achieving interactive visualization at 150 fps on a laptop with processing times of about 10 minutes on a GPU.
Capturing and rendering three-dimensional (3D) objects in real time remain a significant challenge, yet hold substantial potential for applications in augmented reality, digital twin systems, remote collaboration and prototyping. We present an end-to-end pipeline that leverages 3D Gaussian Splatting (3D GS) to enable rapid acquisition and interactive rendering of real-world objects using a mobile device, cloud processing and a local computer. Users scan an object with a smartphone video, upload it for automated 3D reconstruction, and visualize it interactively in Unity at an average of 150 frames per second (fps) on a laptop. The system integrates mobile capture, cloud-based 3D GS and Unity rendering to support real-time telepresence. Our experiments show that the pipeline processes scans in approximately 10 minutes on a graphics processing unit (GPU) achieving real-time rendering on the laptop.