Synthetic Privileged Information Enhances Medical Image Representation Learning
This addresses a bottleneck in medical image analysis for scenarios with scarce or unpaired data, offering a practical solution with significant performance gains.
The paper tackles the problem of limited paired datasets in medical image representation learning by using synthetic generation of paired information, resulting in up to 5.6x error reduction compared to authentic multi-modal paired datasets.
Multimodal self-supervised representation learning has consistently proven to be a highly effective method in medical image analysis, offering strong task performance and producing biologically informed insights. However, these methods heavily rely on large, paired datasets, which is prohibitive for their use in scenarios where paired data does not exist, or there is only a small amount available. In contrast, image generation methods can work well on very small datasets, and can find mappings between unpaired datasets, meaning an effectively unlimited amount of paired synthetic data can be generated. In this work, we demonstrate that representation learning can be significantly improved by synthetically generating paired information, both compared to training on either single-modality (up to 4.4x error reduction) or authentic multi-modal paired datasets (up to 5.6x error reduction).