LMAR: Language Model Augmented Retriever for Domain-specific Knowledge Indexing
This work addresses the challenge of adapting retrieval systems to domain-specific contexts, offering a practical solution for scalable knowledge indexing, though it is incremental as it builds on existing fine-tuning and data augmentation methods.
The paper tackles the problem of domain-specific knowledge retrieval in RAG systems by proposing LMAR, a framework that uses LLM-guided data synthesis and contrastive embedding adaptation, achieving improved performance across multiple benchmark datasets with moderate hardware and low latency.
Retrieval Augmented Generation (RAG) systems often struggle with domain-specific knowledge due to performance deterioration of pre-trained embeddings and prohibitive computational costs of large language model (LLM)-based retrievers. While fine-tuning data augmentation embedding models offers a promising direction, its effectiveness is limited by the need for high-quality training data and reliable chunking strategies that preserve contextual integrity. We propose LMAR (Language Model Augmented Retriever), a model-agnostic framework that addresses these challenges by combining LLM-guided data synthesis with contrastive embedding adaptation and efficient text clustering. LMAR consists of a two-stage pipeline: (1) Triplet sampling and synthetic data augmentation, where LLMs act as both labeler and validator to ensure high-fidelity supervision throughout the pipeline. Experimental results across multiple domain-specific benchmark datasets demonstrate that LMAR outperforms multiple baseline models, while maintaining moderate hardware requirements and low latency. Its model-agnostic nature further enables seamless integration with emerging RAG architectures and text embedding models, ensuring continual improvements without redesigning the pipeline. These results highlight LMAR as a practical and cost-effective solution for scalable domain-specific adaptation.