Towards a General-Purpose Zero-Shot Synthetic Low-Light Image and Video Pipeline
This work addresses the lack of realistic synthetic low-light data for machine learning tasks, enabling better training and evaluation in low-light conditions.
The paper tackles the challenge of generating realistic synthetic low-light images and videos by proposing a Degradation Estimation Network (DEN) that estimates physics-informed noise distributions without camera metadata, achieving improvements of up to 24% KLD, 21% LPIPS, and 62% AP50-95 in tasks like noise replication, video enhancement, and object detection.
Low-light conditions pose significant challenges for both human and machine annotation. This in turn has led to a lack of research into machine understanding for low-light images and (in particular) videos. A common approach is to apply annotations obtained from high quality datasets to synthetically created low light versions. In addition, these approaches are often limited through the use of unrealistic noise models. In this paper, we propose a new Degradation Estimation Network (DEN), which synthetically generates realistic standard RGB (sRGB) noise without the requirement for camera metadata. This is achieved by estimating the parameters of physics-informed noise distributions, trained in a self-supervised manner. This zero-shot approach allows our method to generate synthetic noisy content with a diverse range of realistic noise characteristics, unlike other methods which focus on recreating the noise characteristics of the training data. We evaluate our proposed synthetic pipeline using various methods trained on its synthetic data for typical low-light tasks including synthetic noise replication, video enhancement, and object detection, showing improvements of up to 24\% KLD, 21\% LPIPS, and 62\% AP$_{50-95}$, respectively.