Masks and Manuscripts: Advancing Medical Pre-training with End-to-End Masking and Narrative Structuring
This work addresses semantic variability in medical reports for improved cross-modal representation, but it appears incremental as it builds on existing contrastive learning methods with specific adaptations for the medical domain.
The paper tackles challenges in medical contrastive learning, such as inconsistent semantics and sample pair morphology, by proposing a two-step approach that standardizes text reports into a triplet format and introduces Meijering-based masking for visual pre-training, resulting in new benchmarks in medical image analysis.
Contemporary medical contrastive learning faces challenges from inconsistent semantics and sample pair morphology, leading to dispersed and converging semantic shifts. The variability in text reports, due to multiple authors, complicates semantic consistency. To tackle these issues, we propose a two-step approach. Initially, text reports are converted into a standardized triplet format, laying the groundwork for our novel concept of ``observations'' and ``verdicts''. This approach refines the {Entity, Position, Exist} triplet into binary questions, guiding towards a clear ``verdict''. We also innovate in visual pre-training with a Meijering-based masking, focusing on features representative of medical images' local context. By integrating this with our text conversion method, our model advances cross-modal representation in a multimodal contrastive learning framework, setting new benchmarks in medical image analysis.