RESMatch: Referring Expression Segmentation in a Semi-Supervised Manner
This work addresses the need for reducing data annotation in RES, a crucial task for human-AI interaction, though it is incremental as it adapts existing SSL techniques to a specific domain.
The paper tackles the problem of referring expression segmentation (RES) by introducing RESMatch, the first semi-supervised learning approach for this task, which significantly outperforms baseline methods and sets a new state-of-the-art on multiple datasets.
Referring expression segmentation (RES), a task that involves localizing specific instance-level objects based on free-form linguistic descriptions, has emerged as a crucial frontier in human-AI interaction. It demands an intricate understanding of both visual and textual contexts and often requires extensive training data. This paper introduces RESMatch, the first semi-supervised learning (SSL) approach for RES, aimed at reducing reliance on exhaustive data annotation. Extensive validation on multiple RES datasets demonstrates that RESMatch significantly outperforms baseline approaches, establishing a new state-of-the-art. Although existing SSL techniques are effective in image segmentation, we find that they fall short in RES. Facing the challenges including the comprehension of free-form linguistic descriptions and the variability in object attributes, RESMatch introduces a trifecta of adaptations: revised strong perturbation, text augmentation, and adjustments for pseudo-label quality and strong-weak supervision. This pioneering work lays the groundwork for future research in semi-supervised learning for referring expression segmentation.