HISR: Hybrid Implicit Surface Representation for Photorealistic 3D Human Reconstruction
This work addresses the challenge of accurately modeling human shapes with varying material properties for applications in computer graphics and vision, representing an incremental improvement over existing neural reconstruction methods.
The paper tackles the problem of reconstructing photorealistic 3D human shapes with heterogeneous materials like skin, hair, and clothes by introducing a hybrid implicit surface representation that combines surface-based rendering for opaque regions and volume rendering for translucent regions, achieving state-of-the-art results on 3D human reconstructions and competitive performance on other objects.
Neural reconstruction and rendering strategies have demonstrated state-of-the-art performances due, in part, to their ability to preserve high level shape details. Existing approaches, however, either represent objects as implicit surface functions or neural volumes and still struggle to recover shapes with heterogeneous materials, in particular human skin, hair or clothes. To this aim, we present a new hybrid implicit surface representation to model human shapes. This representation is composed of two surface layers that represent opaque and translucent regions on the clothed human body. We segment different regions automatically using visual cues and learn to reconstruct two signed distance functions (SDFs). We perform surface-based rendering on opaque regions (e.g., body, face, clothes) to preserve high-fidelity surface normals and volume rendering on translucent regions (e.g., hair). Experiments demonstrate that our approach obtains state-of-the-art results on 3D human reconstructions, and also shows competitive performances on other objects.