PhraseStereo: The First Open-Vocabulary Stereo Image Segmentation Dataset
This addresses the problem of limited stereo vision data for researchers in multimodal semantic segmentation, though it is incremental as it builds upon existing datasets.
The authors tackled the lack of stereo image datasets for phrase-region segmentation by introducing PhraseStereo, the first open-vocabulary dataset for this task, generated from existing single-view data to enable research in multimodal learning with depth cues.
Understanding how natural language phrases correspond to specific regions in images is a key challenge in multimodal semantic segmentation. Recent advances in phrase grounding are largely limited to single-view images, neglecting the rich geometric cues available in stereo vision. For this, we introduce PhraseStereo, the first novel dataset that brings phrase-region segmentation to stereo image pairs. PhraseStereo builds upon the PhraseCut dataset by leveraging GenStereo to generate accurate right-view images from existing single-view data, enabling the extension of phrase grounding into the stereo domain. This new setting introduces unique challenges and opportunities for multimodal learning, particularly in leveraging depth cues for more precise and context-aware grounding. By providing stereo image pairs with aligned segmentation masks and phrase annotations, PhraseStereo lays the foundation for future research at the intersection of language, vision, and 3D perception, encouraging the development of models that can reason jointly over semantics and geometry. The PhraseStereo dataset will be released online upon acceptance of this work.