Constructing and Interpreting Digital Twin Representations for Visual Reasoning via Reinforcement Learning
This addresses the need for unified and generalizable visual reasoning models, though it is incremental as it builds on existing reinforcement learning and representation techniques.
The paper tackles the problem of fragmented task-specific models in visual reasoning by proposing DT-R1, a reinforcement learning framework that uses digital twin representations to unify multi-modal visual tasks, achieving consistent improvements over state-of-the-art models across six benchmarks.
Visual reasoning may require models to interpret images and videos and respond to implicit text queries across diverse output formats, from pixel-level segmentation masks to natural language descriptions. Existing approaches rely on supervised fine-tuning with task-specific architectures. For example, reasoning segmentation, grounding, summarization, and visual question answering each demand distinct model designs and training, preventing unified solutions and limiting cross-task and cross-modality generalization. Hence, we propose DT-R1, a reinforcement learning framework that trains large language models to construct digital twin representations of complex multi-modal visual inputs and then reason over these high-level representations as a unified approach to visual reasoning. Specifically, we train DT-R1 using GRPO with a novel reward that validates both structural integrity and output accuracy. Evaluations in six visual reasoning benchmarks, covering two modalities and four task types, demonstrate that DT-R1 consistently achieves improvements over state-of-the-art task-specific models. DT-R1 opens a new direction where visual reasoning emerges from reinforcement learning with digital twin representations.