Turb-Seg-Res: A Segment-then-Restore Pipeline for Dynamic Videos with Atmospheric Turbulence
This addresses the challenge of video restoration in turbulent environments for long-range imaging systems, offering an incremental improvement by adapting existing methods to dynamic scenes.
The paper tackles the problem of restoring videos degraded by atmospheric turbulence in dynamic scenes, presenting a segment-then-restore pipeline that separates dynamic and static components using mean optical flow and unsupervised motion segmentation, then enhances them with turbulence statistics and a transformer model, resulting in restored geometric distortion and enhanced sharpness for videos.
Tackling image degradation due to atmospheric turbulence, particularly in dynamic environment, remains a challenge for long-range imaging systems. Existing techniques have been primarily designed for static scenes or scenes with small motion. This paper presents the first segment-then-restore pipeline for restoring the videos of dynamic scenes in turbulent environment. We leverage mean optical flow with an unsupervised motion segmentation method to separate dynamic and static scene components prior to restoration. After camera shake compensation and segmentation, we introduce foreground/background enhancement leveraging the statistics of turbulence strength and a transformer model trained on a novel noise-based procedural turbulence generator for fast dataset augmentation. Benchmarked against existing restoration methods, our approach restores most of the geometric distortion and enhances sharpness for videos. We make our code, simulator, and data publicly available to advance the field of video restoration from turbulence: riponcs.github.io/TurbSegRes