Assessment of RAG and Fine-Tuning for Industrial Question-Answering-Applications
This work provides practical guidance for enterprises choosing between RAG and FT for domain-specific QA, though the findings are incremental given existing comparisons.
The study compares Retrieval-Augmented Generation (RAG) and fine-tuning (FT) for industrial QA in the automotive domain, finding that RAG is the most effective and cost-efficient adaptation method, enabling open-source models to achieve comparable quality to premium models.
Large Language Models (LLMs) are increasingly employed in enterprise question-answering (QA) systems, requiring adaptation to domain-specific knowledge. Among the most prevalent methods for incorporating such knowledge are Retrieval-Augmented Generation (RAG) and fine-tuning (FT). Yet, from a cost-accuracy trade-off perspective, it remains unclear which approach best suits industry scenarios. This study examines the impact of RAG and FT on two closed datasets specific to the automotive industry, assessing answer quality and operational costs. We extend the Cost-of-Pass framework proposed by Erol et al. (arXiv:2504.13359) to jointly assess output quality, generation cost, and user interaction cost. Our findings reveal that while premium models perform best out of the box, open-source models can achieve comparable quality when enhanced with RAG. Overall, RAG emerges as the most effective and cost-efficient adaptation method for both closed- and open-source models.