DeTurb: Atmospheric Turbulence Mitigation with Deformable 3D Convolutions and 3D Swin Transformers
This addresses the challenge of atmospheric turbulence mitigation for applications like surveillance and astronomy, representing a strong specific gain in this domain.
The paper tackles the problem of atmospheric turbulence in long-range imaging, which degrades scene quality due to spatiotemporal distortions, by proposing a framework that combines geometric restoration with an enhancement module using deformable 3D convolutions and 3D Swin Transformers, achieving superior performance over state-of-the-art methods on synthetic and real data.
Atmospheric turbulence in long-range imaging significantly degrades the quality and fidelity of captured scenes due to random variations in both spatial and temporal dimensions. These distortions present a formidable challenge across various applications, from surveillance to astronomy, necessitating robust mitigation strategies. While model-based approaches achieve good results, they are very slow. Deep learning approaches show promise in image and video restoration but have struggled to address these spatiotemporal variant distortions effectively. This paper proposes a new framework that combines geometric restoration with an enhancement module. Random perturbations and geometric distortion are removed using a pyramid architecture with deformable 3D convolutions, resulting in aligned frames. These frames are then used to reconstruct a sharp, clear image via a multi-scale architecture of 3D Swin Transformers. The proposed framework demonstrates superior performance over the state of the art for both synthetic and real atmospheric turbulence effects, with reasonable speed and model size.