SunStage: Portrait Reconstruction and Relighting using the Sun as a Light Stage
This provides an accessible alternative to expensive light stages for portrait reconstruction and relighting, making it useful for applications in graphics and photography.
The paper tackles the problem of facial reconstruction and relighting by proposing SunStage, a method that uses only a smartphone camera and the sun to capture data comparable to a light stage, enabling detailed facial geometry and appearance reconstruction from an uncalibrated outdoor video.
A light stage uses a series of calibrated cameras and lights to capture a subject's facial appearance under varying illumination and viewpoint. This captured information is crucial for facial reconstruction and relighting. Unfortunately, light stages are often inaccessible: they are expensive and require significant technical expertise for construction and operation. In this paper, we present SunStage: a lightweight alternative to a light stage that captures comparable data using only a smartphone camera and the sun. Our method only requires the user to capture a selfie video outdoors, rotating in place, and uses the varying angles between the sun and the face as guidance in joint reconstruction of facial geometry, reflectance, camera pose, and lighting parameters. Despite the in-the-wild un-calibrated setting, our approach is able to reconstruct detailed facial appearance and geometry, enabling compelling effects such as relighting, novel view synthesis, and reflectance editing. Results and interactive demos are available at https://sunstage.cs.washington.edu/.