Pseudo-Labeling Enhanced by Privileged Information and Its Application to In Situ Sequencing Images
This addresses the challenge of limited ground truth in biological and biomedical vision applications, offering a domain-adaptable solution for semi-supervised learning.
The paper tackles the problem of label-scarce object detection in biological vision domains, such as decoding barcodes from In-Situ-Sequencing images, by proposing a semi-supervised framework that incorporates privileged information into pseudo-labeling, achieving improved performance on both ISS images and the COCO benchmark.
Various strategies for label-scarce object detection have been explored by the computer vision research community. These strategies mainly rely on assumptions that are specific to natural images and not directly applicable to the biological and biomedical vision domains. For example, most semi-supervised learning strategies rely on a small set of labeled data as a confident source of ground truth. In many biological vision applications, however, the ground truth is unknown and indirect information might be available in the form of noisy estimations or orthogonal evidence. In this work, we frame a crucial problem in spatial transcriptomics - decoding barcodes from In-Situ-Sequencing (ISS) images - as a semi-supervised object detection (SSOD) problem. Our proposed framework incorporates additional available sources of information into a semi-supervised learning framework in the form of privileged information. The privileged information is incorporated into the teacher's pseudo-labeling in a teacher-student self-training iteration. Although the available privileged information could be data domain specific, we have introduced a general strategy of pseudo-labeling enhanced by privileged information (PLePI) and exemplified the concept using ISS images, as well on the COCO benchmark using extra evidence provided by CLIP.