A Systematic Evaluation of Retrieval-Augmented Generation and Language Models for Space Operations
For space operations practitioners, this work provides a systematic evaluation of RAG pipelines, though the results are incremental and lack quantitative benchmarks.
This paper systematically evaluates Retrieval-Augmented Generation (RAG) pipelines combining LLMs with retrieval techniques for space operations, showing significant improvements in knowledge access and decision-making. No concrete numbers are provided.
The rapid expansion of space activities has led to an unprecedented accumulation of technical documentation, operational guidelines, and scientific literature, creating challenges for timely decision-making in space operations. Effective management in space operations requires tools capable of efficiently processing vast and heterogeneous information sources. This paper systematically evaluates the performance of Retrieval Augmented Generation (RAG) pipelines, combining Large Language Models (LLMs) with information retrieval techniques for extracting and synthesizing actionable knowledge from domain-specific documents. We compare various retrieval strategies, embedding models, and LLM answers to assess their impact on information accuracy, relevance, and reliability. Our results demonstrate that RAG pipelines can significantly enhance knowledge access, reduce uncertainty, and support decision-making in complex space operations.