MultiTalent: A Multi-Dataset Approach to Medical Image Segmentation
This addresses the need for better segmentation in medical imaging by leveraging available data more effectively, though it is incremental as it builds on existing multi-dataset approaches.
The paper tackles the problem of underutilizing multiple medical imaging datasets with diverse annotations by proposing MultiTalent, a method that trains a single model on multiple CT datasets, resulting in improved segmentation performance, especially for lesions and challenging structures, compared to single-dataset training and other baselines.
The medical imaging community generates a wealth of datasets, many of which are openly accessible and annotated for specific diseases and tasks such as multi-organ or lesion segmentation. Current practices continue to limit model training and supervised pre-training to one or a few similar datasets, neglecting the synergistic potential of other available annotated data. We propose MultiTalent, a method that leverages multiple CT datasets with diverse and conflicting class definitions to train a single model for a comprehensive structure segmentation. Our results demonstrate improved segmentation performance compared to previous related approaches, systematically, also compared to single dataset training using state-of-the-art methods, especially for lesion segmentation and other challenging structures. We show that MultiTalent also represents a powerful foundation model that offers a superior pre-training for various segmentation tasks compared to commonly used supervised or unsupervised pre-training baselines. Our findings offer a new direction for the medical imaging community to effectively utilize the wealth of available data for improved segmentation performance. The code and model weights will be published here: [tba]