Differentiable Adaptive 4D Structured Illumination for Joint Capture of Shape and Reflectance
For computer vision and graphics researchers, this work provides a novel method for efficient joint acquisition of shape and reflectance, though it is incremental as it builds on existing structured light and differentiable rendering concepts.
This paper introduces a differentiable framework for adaptive 4D structured illumination that jointly captures shape and reflectance with a single camera. The method achieves depth results comparable to state-of-the-art techniques and reflectance results validated against photographs.
We present a differentiable framework to adaptively compute 4D illumination conditions with respect to an object, for efficient, high-quality simultaneous acquisition of its shape and reflectance, with a unified spatial-angular structured light and a single camera. Using a simple histogram-based pixel-level probability model for depth and reflectance, we differentiably link the next illumination condition(s) with a loss that encourages the reduction in depth uncertainty. As new structured illumination is cast, corresponding image measurements are used to update the uncertainty at each pixel. Finally, a fine-tuning-based approach reconstructs the depth map and reflectance parameter maps, by minimizing the differences between all physical measurements and their simulated counterparts. The effectiveness of our framework is demonstrated on physical objects with wide variations in shape and appearance. Our depth results compare favorably with state-of-the-art techniques, while our reflectance results are comparable when validated against photographs.