Good Enough: Is it Worth Improving your Label Quality?
This work addresses the cost-benefit trade-off of label quality improvement for medical image segmentation practitioners, providing practical guidance on when such efforts are worthwhile.
The study investigated whether improving label quality in medical image segmentation is beneficial, finding that while higher-quality labels improve in-domain performance, gains are unclear below a small threshold, and label quality has minimal impact on pre-training as models transfer general concepts rather than detailed annotations.
Improving label quality in medical image segmentation is costly, but its benefits remain unclear. We systematically evaluate its impact using multiple pseudo-labeled versions of CT datasets, generated by models like nnU-Net, TotalSegmentator, and MedSAM. Our results show that while higher-quality labels improve in-domain performance, gains remain unclear if below a small threshold. For pre-training, label quality has minimal impact, suggesting that models rather transfer general concepts than detailed annotations. These findings provide guidance on when improving label quality is worth the effort.