Towards 3D Vision with Low-Cost Single-Photon Cameras
This work addresses 3D vision for applications like robotics or scanning by enabling reconstruction with affordable sensors, though it appears incremental in adapting neural rendering to this specific camera type.
The paper tackles 3D shape reconstruction of Lambertian objects using low-cost single-photon cameras, achieving successful recovery of complex shapes from simulated data and demonstrating reconstruction from real-world captures with a commodity sensor.
We present a method for reconstructing 3D shape of arbitrary Lambertian objects based on measurements by miniature, energy-efficient, low-cost single-photon cameras. These cameras, operating as time resolved image sensors, illuminate the scene with a very fast pulse of diffuse light and record the shape of that pulse as it returns back from the scene at a high temporal resolution. We propose to model this image formation process, account for its non-idealities, and adapt neural rendering to reconstruct 3D geometry from a set of spatially distributed sensors with known poses. We show that our approach can successfully recover complex 3D shapes from simulated data. We further demonstrate 3D object reconstruction from real-world captures, utilizing measurements from a commodity proximity sensor. Our work draws a connection between image-based modeling and active range scanning and is a step towards 3D vision with single-photon cameras.