Multimodal Alignment of Histopathological Images Using Cell Segmentation and Point Set Matching for Integrative Cancer Analysis
This method provides a solution for researchers and clinicians to integrate complementary information from different histopathological stains, improving cancer analysis and interpretation.
This paper addresses the challenge of aligning multimodal histopathological images (MxIF and H&E) at the cellular level by treating cells as point sets. It achieves high alignment accuracy on ovarian cancer tissue microarrays, enabling the integration of cell-level features and the generation of virtual H&E images from MxIF data.
Histopathological imaging is vital for cancer research and clinical practice, with multiplexed Immunofluorescence (MxIF) and Hematoxylin and Eosin (H&E) providing complementary insights. However, aligning different stains at the cell level remains a challenge due to modality differences. In this paper, we present a novel framework for multimodal image alignment using cell segmentation outcomes. By treating cells as point sets, we apply Coherent Point Drift (CPD) for initial alignment and refine it with Graph Matching (GM). Evaluated on ovarian cancer tissue microarrays (TMAs), our method achieves high alignment accuracy, enabling integration of cell-level features across modalities and generating virtual H&E images from MxIF data for enhanced clinical interpretation.