Qian He

Zhengyi Liu, Yacheng Tan, Qian He et al.

Convolutional neural networks (CNNs) are good at extracting contexture features within certain receptive fields, while transformers can model the global long-range dependency features. By absorbing the advantage of transformer and the merit of CNN, Swin Transformer shows strong feature representation ability. Based on it, we propose a cross-modality fusion model SwinNet for RGB-D and RGB-T salient object detection. It is driven by Swin Transformer to extract the hierarchical features, boosted by attention mechanism to bridge the gap between two modalities, and guided by edge information to sharp the contour of salient object. To be specific, two-stream Swin Transformer encoder first extracts multi-modality features, and then spatial alignment and channel re-calibration module is presented to optimize intra-level cross-modality features. To clarify the fuzzy boundary, edge-guided decoder achieves inter-level cross-modality fusion under the guidance of edge features. The proposed model outperforms the state-of-the-art models on RGB-D and RGB-T datasets, showing that it provides more insight into the cross-modality complementarity task.

Nouamane Laanait, Qian He, Albina Y. Borisevich

Deep learning has demonstrated superb efficacy in processing imaging data, yet its suitability in solving challenging inverse problems in scientific imaging has not been fully explored. Of immense interest is the determination of local material properties from atomically-resolved imaging, such as electron microscopy, where such information is encoded in subtle and complex data signatures, and whose recovery and interpretation necessitate intensive numerical simulations subject to the requirement of near-perfect knowledge of the experimental setup. We demonstrate that an end-to-end deep learning model can successfully recover 3-dimensional atomic distortions of a variety of oxide perovskite materials from a single 2-dimensional experimental scanning transmission electron (STEM) micrograph, in the process resolving a longstanding question in the recovery of 3-D atomic distortions from STEM experiments. Our results indicate that deep learning is a promising approach to efficiently address unsolved inverse problems in scientific imaging and to underpin novel material investigations at atomic resolution.