MMSearch-R1: Incentivizing LMMs to Search
This addresses the need for more efficient and adaptive search capabilities in LMMs for real-world deployment, representing a novel method rather than an incremental improvement.
The paper tackles the problem of inefficient search behaviors in large multimodal models (LMMs) by introducing MMSearch-R1, an end-to-end reinforcement learning framework that enables on-demand, multi-turn search in real-world Internet environments, resulting in performance matching a larger RAG-based model while reducing search calls by over 30%.
Robust deployment of large multimodal models (LMMs) in real-world scenarios requires access to external knowledge sources, given the complexity and dynamic nature of real-world information. Existing approaches such as retrieval-augmented generation (RAG) and prompt engineered search agents rely on rigid pipelines, often leading to inefficient or excessive search behaviors. We present MMSearch-R1, the first end-to-end reinforcement learning framework that enables LMMs to perform on-demand, multi-turn search in real-world Internet environments. Our framework integrates both image and text search tools, allowing the model to reason about when and how to invoke them guided by an outcome-based reward with a search penalty. To support training, We collect a multimodal search VQA dataset through a semi-automated pipeline that covers diverse visual and textual knowledge needs and curate a search-balanced subset with both search-required and search-free samples, which proves essential for shaping efficient and on-demand search behavior. Extensive experiments on knowledge-intensive and info-seeking VQA tasks show that our model not only outperforms RAG-based baselines of the same model size, but also matches the performance of a larger RAG-based model while reducing search calls by over 30%. We further analyze key empirical findings to offer actionable insights for advancing research in multimodal search.