Efficient Image Annotation via Semi-Supervised Object Segmentation with Label Propagation
For service robotics applications requiring rapid deployment, this method reduces annotation effort for object segmentation across 50 classes.
The paper presents a semi-supervised label propagation method for household object segmentation that uses an ensemble of Hopfield networks to assign labels from foundation model embeddings, achieving automatic labeling of 60% of data in a RoboCup@Home setting with limited annotation overhead.
Reliable object perception is necessary for general-purpose service robots. Open-vocabulary detectors struggle to generalize beyond a few classes and fully supervised training of object detectors requires time-intensive annotations. We present a semi-supervised label propagation approach for household object segmentation. A segment proposer generates class-agnostic masks, and an ensemble of Hopfield networks assigns labels by learning representative embeddings in complementary foundation model embedding spaces (CLIP, ViT, Theia). Our approach scales to 50 object classes with limited annotation overhead and can automatically label 60% of the data in a RoboCup@Home setting, where preparation time is severely constrained. Dataset and code are publicly available at https://github.com/ais-bonn/label_propagation.