Neural Acquisition & Representation of Subsurface Scattering
For computer graphics and rendering, this provides a practical way to capture and relight subsurface scattering effects from real objects.
The paper presents a method for acquiring and estimating subsurface scattering properties using a U-Net CNN and 3D scanning, achieving relit images nearly identical to real-world captures. The model generalizes to unseen materials.
We present a method to acquire and estimate the sub-surface scattering properties of light transport at a highly detailed level by learning the pixel footprint response at each point on the object surface. The reconstruction leverages 3D scanning techniques as input to a U-Net CNN. A stereo projector-camera setup using phase-shifted profilometry (PSP) patterns efficiently captures the data for a variety of scattering objects. Reconstructing dense pixel footprints allows for relighting with arbitrary high-resolution projector patterns. The final output is a relit color image. Qualitative and quantitative comparison against illuminated real-world captured images demonstrate that the predicted footprints are almost identical to the actual responses. The same model is trained for multiple views across multiple objects such that the learned representations can be used to generalize to unseen sub-surface scattering materials as well.