Towards Automated Ontology Generation from Unstructured Text: A Multi-Agent LLM Approach
For knowledge engineers, this work provides a controlled study showing that planning-first, artifact-driven multi-agent architectures outperform single-agent LLMs in ontology generation, though improvements are incremental.
The paper tackles automated ontology generation from unstructured text, finding that a multi-agent LLM approach significantly improves structural quality and modestly enhances queryability over a single-agent baseline, with gains driven primarily by front-loaded planning.
Automatically generating formal ontologies from unstructured natural language remains a central challenge in knowledge engineering. While large language models (LLMs) show promise, it remains unclear which architectural design choices drive generation quality and why current approaches fail. We present a controlled experimental study using domain-specific insurance contracts to investigate these questions. We first establish a single-agent LLM baseline, identifying key failure modes such as poor Ontology Design Pattern compliance, structural redundancy, and ineffective iterative repair. We then introduce a multi-agent architecture that decomposes ontology construction into four artifact-driven roles: Domain Expert, Manager, Coder, and Quality Assurer. We evaluate performance across architectural quality (via a panel of heterogeneous LLM judges) and functional usability (via competency question driven SPARQL evaluation with complementary retrieval augmented generation based assessment). Results show that the multi-agent approach significantly improves structural quality and modestly enhances queryability, with gains driven primarily by front-loaded planning. These findings highlight planning-first, artifact-driven generation as a promising and more auditable path toward scalable automated ontology engineering.