Fine-tuning Large Enterprise Language Models via Ontological Reasoning
This work addresses the challenge of domain orientation in LLMs for enterprise applications, offering a novel integration method that could enhance task-specific performance, though it appears incremental in combining existing techniques.
The paper tackles the problem of fine-tuning large language models (LLMs) for enterprise tasks by addressing the lack of domain-specific knowledge in training data, proposing a neurosymbolic architecture that uses ontological reasoning from Enterprise Knowledge Graphs (EKGs) to build specialized corpora for improved adaptation.
Large Language Models (LLMs) exploit fine-tuning as a technique to adapt to diverse goals, thanks to task-specific training data. Task specificity should go hand in hand with domain orientation, that is, the specialization of an LLM to accurately address the tasks of a given realm of interest. However, models are usually fine-tuned over publicly available data or, at most, over ground data from databases, ignoring business-level definitions and domain experience. On the other hand, Enterprise Knowledge Graphs (EKGs) are able to capture and augment such domain knowledge via ontological reasoning. With the goal of combining LLM flexibility with the domain orientation of EKGs, we propose a novel neurosymbolic architecture that leverages the power of ontological reasoning to build task- and domain-specific corpora for LLM fine-tuning.