NECA: Neural Customizable Human Avatar
This work addresses the need for versatile and editable human avatars in 3D applications, representing an incremental improvement over existing methods.
The paper tackles the problem of creating fully customizable human avatars from monocular or sparse-view videos, achieving photorealistic rendering with high-frequency details and enabling editing tasks like novel pose synthesis and relighting.
Human avatar has become a novel type of 3D asset with various applications. Ideally, a human avatar should be fully customizable to accommodate different settings and environments. In this work, we introduce NECA, an approach capable of learning versatile human representation from monocular or sparse-view videos, enabling granular customization across aspects such as pose, shadow, shape, lighting and texture. The core of our approach is to represent humans in complementary dual spaces and predict disentangled neural fields of geometry, albedo, shadow, as well as an external lighting, from which we are able to derive realistic rendering with high-frequency details via volumetric rendering. Extensive experiments demonstrate the advantage of our method over the state-of-the-art methods in photorealistic rendering, as well as various editing tasks such as novel pose synthesis and relighting. The code is available at https://github.com/iSEE-Laboratory/NECA.