Seeing the Unseen in Low-light Spike Streams
This work addresses a domain-specific challenge in neuromorphic vision for high-speed applications, representing an incremental advancement in spike stream reconstruction methods.
The paper tackles the problem of reconstructing perceptible images from low-light, high-speed spike camera streams, which suffer from severe noise and sparse information, by proposing Diff-SPK, a diffusion-based method that leverages generative priors to supplement texture, achieving improved reconstruction quality under diverse low-light conditions.
Spike camera, a type of neuromorphic sensor with high-temporal resolution, shows great promise for high-speed visual tasks. Unlike traditional cameras, spike camera continuously accumulates photons and fires asynchronous spike streams. Due to unique data modality, spike streams require reconstruction methods to become perceptible to the human eye. However, lots of methods struggle to handle spike streams in low-light high-speed scenarios due to severe noise and sparse information. In this work, we propose Diff-SPK, a diffusion-based reconstruction method. Diff-SPK effectively leverages generative priors to supplement texture information under diverse low-light conditions. Specifically, it first employs an Enhanced Texture from Inter-spike Interval (ETFI) to aggregate sparse information from low-light spike streams. Then, the encoded ETFI by a suitable encoder serve as the input of ControlNet for high-speed scenes generation. To improve the quality of results, we introduce an ETFI-based feature fusion module during the generation process.