Deep histological synthesis from mass spectrometry imaging for multimodal registration
This work addresses a domain-specific problem for researchers in medical imaging by providing an incremental improvement in multimodal registration.
The paper tackled the challenge of registering histological and mass spectrometry imaging (MSI) by synthesizing histological images from MSI using a pix2pix model, achieving increases in mutual information (MI) and structural similarity index (SSIM) of +0.924 and +0.419 compared to a baseline U-Net model.
Registration of histological and mass spectrometry imaging (MSI) allows for more precise identification of structural changes and chemical interactions in tissue. With histology and MSI having entirely different image formation processes and dimensionalities, registration of the two modalities remains an ongoing challenge. This work proposes a solution that synthesises histological images from MSI, using a pix2pix model, to effectively enable unimodal registration. Preliminary results show promising synthetic histology images with limited artifacts, achieving increases in mutual information (MI) and structural similarity index measures (SSIM) of +0.924 and +0.419, respectively, compared to a baseline U-Net model. Our source code is available on GitHub: https://github.com/kimberley/MIUA2025.