Carlos Aliaga

Shizhan Zhu, Shunsuke Saito, Aljaz Bozic et al.

Image-based lighting (IBL) is a widely used technique that renders objects using a high dynamic range image or environment map. However, aggregating the irradiance at the object's surface is computationally expensive, in particular for non-opaque, translucent materials that require volumetric rendering techniques. In this paper we present a fast neural 3D reconstruction and relighting model that extends volumetric implicit models such as neural radiance fields to be relightable using IBL. It is general enough to handle materials that exhibit complex light transport effects, such as translucency and glossy reflections from detailed surface geometry, producing realistic and compelling results. Rendering can be within a second at 800$\times$800 resolution (0.72s on an NVIDIA 3090 GPU and 0.30s on an A100 GPU) without engineering optimization. Our code and dataset are available at https://zhusz.github.io/TRHM-Webpage/.

Carlos Aliaga, Adrian Jarabo

In this paper we present a spectral optical inversion for skin for path tracing-based rendering of subsurface scattering. Skin is a complex multilayered medium, with appearance determined by the mixture of biophysical chromophores. However, current methods rely on medium homogeneization, with optical parameters obtained via albedo inversion from a reflectance texture and hand-tuned scattering distance and anisotropy. This results into significant art-skilled manual labor for authoring, and an inaccurate scattering profile for skin. To solve these problems, we generalize existing albedo inversion techniques, and propose a framework that predicts full-spectral skin scattering parameters from a single RGB diffuse albedo. Our method builds upon a new mixture-of-media representation, that approximates the aggregated multilayered appearance of skin by mixing the aggregated scattering of three uncorrelated media. We train a chained neural decoder that maps RGB diffuse albedo to the optical properties of the mixture of media, including anisotropy, scattering radius and scattering albedo. Then, we show this mixture can be used in a random-walk-based path tracer with minimal modifications, by simply randomly selecting the medium to traverse.

Carlos Aliaga, Christophe Hery, Mengqi Xia

We present a new method to reconstruct and manipulate the spectral properties of human skin from simple RGB albedo captures. To this end, we leverage Monte Carlo light simulation over an accurate biophysical human skin layering model parameterized by its most important components, thereby covering a plausible range of colors. The practical complexity of the model allows us to learn the inverse mapping from any albedo to its most probable associated skin properties. Our technique can faithfully reproduce any skin type, being expressive enough to automatically handle more challenging areas like the lips or imperfections in the face. Thanks to the smoothness of the skin parameters maps recovered, the albedo can be robustly edited through meaningful biophysical properties.