Discriminative Perception via Anchored Description for Reasoning Segmentation
This addresses the issue of verbose and inaccurate reasoning in multimodal large language models for segmentation, offering an incremental improvement with specific gains in efficiency and performance.
The paper tackles the problem of reasoning segmentation models generating unfocused reasoning chains by introducing Discriminative Perception via Anchored Description (DPAD), which compels the model to generate descriptive captions to discriminate targets from context, resulting in a 3.09% increase in cIoU on ReasonSeg and a 42% reduction in reasoning chain length.
Reasoning segmentation increasingly employs reinforcement learning to generate explanatory reasoning chains that guide Multimodal Large Language Models. While these geometric rewards are primarily confined to guiding the final localization, they are incapable of discriminating whether the reasoning process remains anchored on the referred region or strays into irrelevant context. Lacking this discriminative guidance, the model's reasoning often devolves into unfocused and verbose chains that ultimately fail to disambiguate and perceive the target in complex scenes. This suggests a need to complement the RL objective with Discriminative Perception, an ability to actively distinguish a target from its context. To realize this, we propose DPAD to compel the model to generate a descriptive caption of the referred object, which is then used to explicitly discriminate by contrasting the caption's semantic relevance to the referred object against the wider context. By optimizing for this discriminative capability, the model is forced to focus on the unique attributes of the target, leading to a more converged and efficient reasoning chain. The descriptive caption also serves as an interpretability rationale that aligns with the segmentation. Experiments on the benchmarks confirm the validity of our approach, delivering substantial performance gains, with the cIoU on ReasonSeg increasing by 3.09% and the reasoning chain length decreasing by approximately 42%. Code is available at https://github.com/mrazhou/DPAD