OpenDenoising: an Extensible Benchmark for Building Comparative Studies of Image Denoisers
This provides a benchmarking tool for researchers and practitioners to evaluate denoisers, addressing a gap in comparative studies, though it is incremental as it builds on existing methods.
The paper tackles the difficulty of comparing image denoisers in real-world conditions by proposing an extensible benchmark tool, showing that MWCNN outperforms other methods for real-world noise with a 60% Kendall's Tau correlation across noise types.
Image denoising has recently taken a leap forward due to machine learning. However, image denoisers, both expert-based and learning-based, are mostly tested on well-behaved generated noises (usually Gaussian) rather than on real-life noises, making performance comparisons difficult in real-world conditions. This is especially true for learning-based denoisers which performance depends on training data. Hence, choosing which method to use for a specific denoising problem is difficult. This paper proposes a comparative study of existing denoisers, as well as an extensible open tool that makes it possible to reproduce and extend the study. MWCNN is shown to outperform other methods when trained for a real-world image interception noise, and additionally is the second least compute hungry of the tested methods. To evaluate the robustness of conclusions, three test sets are compared. A Kendall's Tau correlation of only 60% is obtained on methods ranking between noise types, demonstrating the need for a benchmarking tool.