LightHeadEd: Relightable & Editable Head Avatars from a Smartphone
This addresses the inaccessibility of high-quality head avatar creation for widespread adoption, though it appears incremental as it builds on existing parametric models and neural synthesis techniques.
The paper tackles the problem of creating photorealistic, animatable, and relightable 3D head avatars by introducing a cost-effective method using only a smartphone with polaroid filters, achieving high-quality results without expensive Lightstage setups.
Creating photorealistic, animatable, and relightable 3D head avatars traditionally requires expensive Lightstage with multiple calibrated cameras, making it inaccessible for widespread adoption. To bridge this gap, we present a novel, cost-effective approach for creating high-quality relightable head avatars using only a smartphone equipped with polaroid filters. Our approach involves simultaneously capturing cross-polarized and parallel-polarized video streams in a dark room with a single point-light source, separating the skin's diffuse and specular components during dynamic facial performances. We introduce a hybrid representation that embeds 2D Gaussians in the UV space of a parametric head model, facilitating efficient real-time rendering while preserving high-fidelity geometric details. Our learning-based neural analysis-by-synthesis pipeline decouples pose and expression-dependent geometrical offsets from appearance, decomposing the surface into albedo, normal, and specular UV texture maps, along with the environment maps. We collect a unique dataset of various subjects performing diverse facial expressions and head movements.