Evaluation of Chunking Strategies for Effective Text Embedding in Low-Resource Language on Agricultural Documents
For researchers working on retrieval-augmented generation in low-resource languages, this provides empirical guidance on chunking strategies, though the evaluation is limited to 18 QA pairs.
This study compares four text chunking strategies for RAG on Khmer agricultural documents, finding that character-based Recursive chunking (300 chars) achieves best retrieval and relevance scores (L2 distance 0.4295, Answer Relevance 0.8663, Khmer IoU 0.6441), with statistically significant improvement over Sentence-Based chunking.
In this study, we compare the performance of four text chunking approaches: Recursive, Khmer-Aware, Sentence-Based, and LLM-Based within a Retrieval-Augmented Generation (RAG) framework applied to Khmer agricultural documents. The document chunks are encoded using the BGE-M3 multilingual embedding model and retrieved using the FAISS library. Performance is evaluated using four metrics: Average Retrieval Score (L2 distance), Answer Relevance, Khmer Coverage, and Khmer Intersection over Union, all measured against ground-truth question-answer pairs. For evaluation, we perform 5-fold cross-validation over 18 question-answer pairs. We observe the best performance for the character-based Recursive chunking method with a chunk size of 300 characters, achieving the lowest L2 distance (0.4295 +- 0.0461), highest Answer Relevance (0.8663 +- 0.0199), and highest Khmer IoU (0.6441 +- 0.0347). A paired t-test shows a statistically significant improvement over the Sentence-Based chunking method in L2 distance (p = 0.0121). These results highlight the importance of segmentation granularity and structural preservation for optimizing dense retrieval in morphologically complex, low-resource languages such as Khmer.