Looping Back to Move Forward: Recursive Transformers for Efficient and Flexible Large Multimodal Models
This work addresses efficiency and flexibility for deploying large multimodal models, though it is incremental as it builds on existing recursive and Transformer methods.
The paper tackled the underutilization of parameters in Large Multimodal Models by proposing RecursiveVLM, a recursive Transformer architecture that reuses parameters through recursive refinement, achieving gains of +3% over standard Transformers and +7% over vanilla recursive baselines.
Large Multimodal Models (LMMs) have achieved remarkable success in vision-language tasks, yet their vast parameter counts are often underutilized during both training and inference. In this work, we embrace the idea of looping back to move forward: reusing model parameters through recursive refinement to extract stronger multimodal representations without increasing model size. We propose RecursiveVLM, a recursive Transformer architecture tailored for LMMs. Two key innovations enable effective looping: (i) a Recursive Connector that aligns features across recursion steps by fusing intermediate-layer hidden states and applying modality-specific projections, respecting the distinct statistical structures of vision and language tokens; (ii) a Monotonic Recursion Loss that supervises every step and guarantees performance improves monotonically with recursion depth. This design transforms recursion into an on-demand refinement mechanism: delivering strong results with few loops on resource-constrained devices and progressively improving outputs when more computation resources are available. Experiments show consistent gains of +3% over standard Transformers and +7% over vanilla recursive baselines, demonstrating that strategic looping is a powerful path toward efficient, deployment-adaptive LMMs.