DeLux: Cross-Modal Local Artifact Restoration in Video Using Neuromorphic Data
For computer vision applications requiring robust video under challenging lighting, DeLux offers a novel cross-modal approach that significantly outperforms existing RGB-only and event-guided methods.
DeLux uses neuromorphic event streams to detect and inpaint lighting artifacts in RGB video, achieving over 0.99 MS-SSIM on synthetic benchmarks and up to 88% reduction in artifact severity in real-world automotive footage.
Conventional RGB cameras suffer from lighting artifacts such as flare, glare, flicker, and overexposure, leading to irrecoverable information loss that necessitates computational restoration. However, existing approaches treat these problems in isolation, failing to recover structural details completely obscured by complex spatially discrete image degradations. In this paper, we propose a novel cross-modal restoration paradigm and present DeLux, a modular proof-of-concept pipeline that leverages neuromorphic event streams as a structural prior to guide the targeted detection and inpainting of lighting artifacts in RGB video. Validation on synthetic benchmarks and real-world automotive footage demonstrates that DeLux effectively suppresses local artifacts and restores affected regions. The proposed approach outperforms existing RGB-only baselines and event-guided HDR models, achieving an average MS-SSIM of over 0.99 across all artifact types and demonstrating up to an 88% reduction in artifact severity in real-world automotive footage. The synthetic artifact generation tools and curated real-world evaluation datasets are made publicly available to foster future research on cross-modal restoration.