Development and Evaluation of a Retrieval-Augmented Generation Tool for Creating SAPPhIRE Models of Artificial Systems
This work addresses the problem of reducing manual effort for experts in design-by-analogy by automating knowledge extraction from technical documents, though it appears incremental as it builds on existing RAG and LLM methods.
The researchers tackled the effort-intensive process of creating SAPPhIRE causality models for artificial systems by developing a Retrieval-Augmented Generation (RAG) tool that leverages Large Language Models to generate structured descriptions, and they reported preliminary evaluation results on the tool's factual accuracy and reliability.
Representing systems using the SAPPhIRE causality model is found useful in supporting design-by-analogy. However, creating a SAPPhIRE model of artificial or biological systems is an effort-intensive process that requires human experts to source technical knowledge from multiple technical documents regarding how the system works. This research investigates how to leverage Large Language Models (LLMs) in creating structured descriptions of systems using the SAPPhIRE model of causality. This paper, the second part of the two-part research, presents a new Retrieval-Augmented Generation (RAG) tool for generating information related to SAPPhIRE constructs of artificial systems and reports the results from a preliminary evaluation of the tool's success - focusing on the factual accuracy and reliability of outcomes.