MedKCO: Medical Vision-Language Pretraining via Knowledge-Driven Cognitive Orchestration
This work addresses the challenge of improving generalization in medical VLP models for healthcare applications, representing an incremental advancement through novel curriculum learning and loss adjustments.
The authors tackled the problem of suboptimal feature representations in medical vision-language pretraining (VLP) models by proposing MedKCO, which uses a knowledge-driven cognitive orchestration approach with a two-level curriculum and self-paced asymmetric contrastive loss, resulting in significant performance improvements over baselines in multiple downstream tasks.
Medical vision-language pretraining (VLP) models have recently been investigated for their generalization to diverse downstream tasks. However, current medical VLP methods typically force the model to learn simple and complex concepts simultaneously. This anti-cognitive process leads to suboptimal feature representations, especially under distribution shift. To address this limitation, we propose a Knowledge-driven Cognitive Orchestration for Medical VLP (MedKCO) that involves both the ordering of the pretraining data and the learning objective of vision-language contrast. Specifically, we design a two level curriculum by incorporating diagnostic sensitivity and intra-class sample representativeness for the ordering of the pretraining data. Moreover, considering the inter-class similarity of medical images, we introduce a self-paced asymmetric contrastive loss to dynamically adjust the participation of the pretraining objective. We evaluate the proposed pretraining method on three medical imaging scenarios in multiple vision-language downstream tasks, and compare it with several curriculum learning methods. Extensive experiments show that our method significantly surpasses all baselines. https://github.com/Mr-Talon/MedKCO.