Open-Ended CT Volume Segmentation with Weak Supervision from Language
This work addresses the problem of reducing annotation cost for CT segmentation by leveraging weak supervision from reports, showing significant gains especially when labeled data is scarce.
The authors propose a method for text-conditioned CT volume segmentation that combines voxel-level supervision with slice-level weak supervision from radiology reports. On the ReXGroundingCT dataset, this approach improves dice scores by 8% relative with 1000 fully labeled volumes and 22% with 250 fully labeled volumes.
We introduce a method for training a text-conditioned segmentation model for CT scans, which combines voxel-level supervision with coarse but scalable slice-level supervision from reports. We extract, from a large database of scan-report pairs, descriptions of findings with indices of slices where those findings occur. We then finetune a general-purpose 2D image segmentation model, SAM3, with standard segmentation losses from strongly labeled data and with a slice-level classification loss from the extracted weak supervision. Our results on the ReXGroundingCT dataset illustrate that this strategy improves the segmentation dice score: from an 8% relative gain when there are 1000 fully labeled volumes to 22% when there are 250 fully labeled volumes.