Chunk Twice, Embed Once: A Systematic Study of Segmentation and Representation Trade-offs in Chemistry-Aware Retrieval-Augmented Generation
It addresses foundational design choices for building effective RAG systems in high-stakes chemistry domains, providing actionable guidelines, though it is incremental as it focuses on optimizing existing methods rather than introducing new paradigms.
This study systematically evaluated chunking strategies and embedding models for chemistry-focused Retrieval-Augmented Generation (RAG) systems, finding that recursive token-based chunking (R100-0) and retrieval-optimized embeddings like Nomic and Intfloat E5 variants outperformed other methods, with specific benchmarks showing substantial gains.
Retrieval-Augmented Generation (RAG) systems are increasingly vital for navigating the ever-expanding body of scientific literature, particularly in high-stakes domains such as chemistry. Despite the promise of RAG, foundational design choices -- such as how documents are segmented and represented -- remain underexplored in domain-specific contexts. This study presents the first large-scale, systematic evaluation of chunking strategies and embedding models tailored to chemistry-focused RAG systems. We investigate 25 chunking configurations across five method families and evaluate 48 embedding models on three chemistry-specific benchmarks, including the newly introduced QuestChemRetrieval dataset. Our results reveal that recursive token-based chunking (specifically R100-0) consistently outperforms other approaches, offering strong performance with minimal resource overhead. We also find that retrieval-optimized embeddings -- such as Nomic and Intfloat E5 variants -- substantially outperform domain-specialized models like SciBERT. By releasing our datasets, evaluation framework, and empirical benchmarks, we provide actionable guidelines for building effective and efficient chemistry-aware RAG systems.