Multi-view analysis of unregistered medical images using cross-view transformers
This addresses the challenge of multi-view analysis in medical imaging where registration is not possible, potentially improving diagnostic accuracy for conditions like breast cancer or lung diseases.
The paper tackled the problem of combining information from unregistered multi-view medical images by introducing a cross-view transformer method that transfers information at the spatial feature map level, outperforming baseline models that join feature vectors after global pooling on mammography and chest X-ray datasets.
Multi-view medical image analysis often depends on the combination of information from multiple views. However, differences in perspective or other forms of misalignment can make it difficult to combine views effectively, as registration is not always possible. Without registration, views can only be combined at a global feature level, by joining feature vectors after global pooling. We present a novel cross-view transformer method to transfer information between unregistered views at the level of spatial feature maps. We demonstrate this method on multi-view mammography and chest X-ray datasets. On both datasets, we find that a cross-view transformer that links spatial feature maps can outperform a baseline model that joins feature vectors after global pooling.