Temporal Spatial-Adaptive Interpolation with Deformable Refinement for Electron Microscopic Images
This work addresses frame interpolation for EM images, a domain-specific problem with incremental improvements in handling deformation and quality issues.
The paper tackles the problem of video frame interpolation for electron microscopic (EM) images, which suffer from unstable quality and deformation, by proposing a novel framework that progressively synthesizes interpolated features, achieving superior performance compared to previous methods.
Recently, flow-based methods have achieved promising success in video frame interpolation. However, electron microscopic (EM) images suffer from unstable image quality, low PSNR, and disorderly deformation. Existing flow-based interpolation methods cannot precisely compute optical flow for EM images since only predicting each position's unique offset. To overcome these problems, we propose a novel interpolation framework for EM images that progressively synthesizes interpolated features in a coarse-to-fine manner. First, we extract missing intermediate features by the proposed temporal spatial-adaptive (TSA) interpolation module. The TSA interpolation module aggregates temporal contexts and then adaptively samples the spatial-related features with the proposed residual spatial adaptive block. Second, we introduce a stacked deformable refinement block (SDRB) further enhance the reconstruction quality, which is aware of the matching positions and relevant features from input frames with the feedback mechanism. Experimental results demonstrate the superior performance of our approach compared to previous works, both quantitatively and qualitatively.