Towards Spectral Estimation from a Single RGB Image in the Wild
This work addresses spectral estimation for applications in computer vision and imaging, but it is incremental as it builds on existing deep learning methods for sensitivity function estimation and spectral reconstruction.
The paper tackles the problem of estimating the spectrum from a single RGB image taken under unconstrained conditions, achieving state-of-the-art competitive results on standard benchmarks like ICVL, CAVE, NUS, and NTIRE.
In contrast to the current literature, we address the problem of estimating the spectrum from a single common trichromatic RGB image obtained under unconstrained settings (e.g. unknown camera parameters, unknown scene radiance, unknown scene contents). For this we use a reference spectrum as provided by a hyperspectral image camera, and propose efficient deep learning solutions for sensitivity function estimation and spectral reconstruction from a single RGB image. We further expand the concept of spectral reconstruction such that to work for RGB images taken in the wild and propose a solution based on a convolutional network conditioned on the estimated sensitivity function. Besides the proposed solutions, we study also generic and sensitivity specialized models and discuss their limitations. We achieve state-of-the-art competitive results on the standard example-based spectral reconstruction benchmarks: ICVL, CAVE, NUS and NTIRE. Moreover, our experiments show that, for the first time, accurate spectral estimation from a single RGB image in the wild is within our reach.