MXR-U-Nets for Real Time Hyperspectral Reconstruction
This enables real-time hyperspectral reconstruction for video applications, but it is incremental as it builds on existing CNN work.
The paper tackles hyperspectral image reconstruction from RGB images using a CNN architecture, achieving real-time video capability with a shallower model that has 10% memory footprint, 3x faster inference, and only about a 0.5% performance decrease.
In recent times, CNNs have made significant contributions to applications in image generation, super-resolution and style transfer. In this paper, we build upon the work of Howard and Gugger, He et al. and Misra, D. and propose a CNN architecture that accurately reconstructs hyperspectral images from their RGB counterparts. We also propose a much shallower version of our best model with a 10% relative memory footprint and 3x faster inference, thus enabling real-time video applications while still experiencing only about a 0.5% decrease in performance.