UniIR: Training and Benchmarking Universal Multimodal Information Retrievers
This work addresses the need for versatile retrieval systems for diverse user queries, representing a novel method for a known bottleneck in multimodal information retrieval.
The paper tackles the problem of limited applicability in information retrieval models by introducing UniIR, a unified instruction-guided multimodal retriever that handles eight distinct retrieval tasks across modalities, achieving robust performance across datasets and zero-shot generalization to new tasks.
Existing information retrieval (IR) models often assume a homogeneous format, limiting their applicability to diverse user needs, such as searching for images with text descriptions, searching for a news article with a headline image, or finding a similar photo with a query image. To approach such different information-seeking demands, we introduce UniIR, a unified instruction-guided multimodal retriever capable of handling eight distinct retrieval tasks across modalities. UniIR, a single retrieval system jointly trained on ten diverse multimodal-IR datasets, interprets user instructions to execute various retrieval tasks, demonstrating robust performance across existing datasets and zero-shot generalization to new tasks. Our experiments highlight that multi-task training and instruction tuning are keys to UniIR's generalization ability. Additionally, we construct the M-BEIR, a multimodal retrieval benchmark with comprehensive results, to standardize the evaluation of universal multimodal information retrieval.