Can Structured Data Reduce Epistemic Uncertainty?
This work addresses the issue of unreliable outputs in AI models, particularly for tasks requiring accurate and contextually relevant information, but it appears incremental as it builds on existing ontology and retrieval-augmented generation methods.
The paper tackles the problem of epistemic uncertainty in deep learning by using ontology alignment to improve model learning and reduce hallucination in large language models, resulting in an 8.97% increase in contextual similarity, a 1% increase in factual accuracy, and a 4.847% reduction in hallucination.
In this work, we present a framework that utilizes ontology alignment to improve the learning process of deep learning models. With this approach we show that models fine-tuned using ontologies learn a downstream task at a higher rate with better performance on a sequential classification task compared to the native version of the model. Additionally, we extend our work to showcase how subsumption mappings retrieved during the process of ontology alignment can help enhance Retrieval-Augmented Generation in Large Language Models. The results show that the responses obtained by using subsumption mappings show an increase of 8.97% in contextual similarity and a 1% increase in factual accuracy. We also use these scores to define our Hallucination Index and show that this approach reduces hallucination in LLMs by 4.847%.