Dancing under the stars: video denoising in starlight
This enables photorealistic video in starlight, addressing a challenge in low-light imaging for applications like surveillance or astronomy.
The paper tackled the problem of capturing photorealistic video under extremely low light conditions (starlight, <0.001 lux) by developing a GAN-tuned physics-based noise model and training a video denoiser, achieving improved video quality at the lowest light levels for the first time.
Imaging in low light is extremely challenging due to low photon counts. Using sensitive CMOS cameras, it is currently possible to take videos at night under moonlight (0.05-0.3 lux illumination). In this paper, we demonstrate photorealistic video under starlight (no moon present, $<$0.001 lux) for the first time. To enable this, we develop a GAN-tuned physics-based noise model to more accurately represent camera noise at the lowest light levels. Using this noise model, we train a video denoiser using a combination of simulated noisy video clips and real noisy still images. We capture a 5-10 fps video dataset with significant motion at approximately 0.6-0.7 millilux with no active illumination. Comparing against alternative methods, we achieve improved video quality at the lowest light levels, demonstrating photorealistic video denoising in starlight for the first time.