Cross-modal Attention for MRI and Ultrasound Volume Registration
This work addresses a domain-specific challenge in medical imaging for improved prostate cancer diagnosis, representing an incremental advancement by integrating self-attention into CNNs.
The paper tackles the problem of accurately fusing MRI and ultrasound images for prostate cancer biopsy by developing a cross-modal attention block for image registration, which outperforms a CNN network 10 times its size.
Prostate cancer biopsy benefits from accurate fusion of transrectal ultrasound (TRUS) and magnetic resonance (MR) images. In the past few years, convolutional neural networks (CNNs) have been proved powerful in extracting image features crucial for image registration. However, challenging applications and recent advances in computer vision suggest that CNNs are quite limited in its ability to understand spatial correspondence between features, a task in which the self-attention mechanism excels. This paper aims to develop a self-attention mechanism specifically for cross-modal image registration. Our proposed cross-modal attention block effectively maps each of the features in one volume to all features in the corresponding volume. Our experimental results demonstrate that a CNN network designed with the cross-modal attention block embedded outperforms an advanced CNN network 10 times of its size. We also incorporated visualization techniques to improve the interpretability of our network. The source code of our work is available at https://github.com/DIAL-RPI/Attention-Reg .