VitaminP: cross-modal learning enables whole-cell segmentation from routine histology
For computational pathology and spatial omics, this provides a cost-effective method to obtain whole-cell segmentation from widely available H&E stains, overcoming the limitation of nuclear-only analysis.
VitaminP enables whole-cell segmentation from routine H&E histology by learning from paired H&E-multiplex immunofluorescence data, outperforming four state-of-the-art methods on 14 public datasets covering 34 cancer types and over 7 million instances, and generalizing to unseen datasets including 24 rare cancer types.
Accurate whole-cell and nuclear segmentation is essential for precision pathology and spatial omics, yet routine hematoxylin and eosin (H&E) staining provides limited cytoplasmic contrast, restricting analyses to nuclei. Multiplex immunofluorescence (mIF) facilitates precise whole-cell delineation but remains constrained by cost and accessibility. We introduce VitaminP, a cross-modal learning framework enabling whole cell segmentation from H&E images. By learning from paired H&E-mIF data, VitaminP transfers molecular boundary information from mIF to overcome cytoplasmic contrast in H&E, establishing cross-modal supervision as a general strategy for recovering missing biological structure. We train VitaminP on 14 public datasets covering 34 cancer types and over 7 million instances, integrating publicly available labels with extensive annotations generated in this study, forming one of the largest resources for segmentation. VitaminP outperforms four state-of-the-art methods and generalizes to unseen datasets, including an in-house dataset spanning 24 rare cancer types. We further developed VitaminPScope, an open-source platform providing an interface for scalable inference and enabling broad adoption.