All Relations Lead to Rome: Automated Knowledge Graph Creation and Question Generation
For researchers in hybrid retrieval and question answering, this provides a unified benchmark that combines knowledge graphs and text, but the contribution is incremental as it applies existing methods to a new domain.
The authors introduce ARLtR, a framework for automated knowledge graph construction and fact-grounded question-answer generation, creating a historical dataset with over 19,000 entities, 16,000 chunks, and 8,400 QA pairs that unifies symbolic graph and dense retrieval representations.
Large language models have substantially improved information retrieval and question answering; however, existing datasets generally support either vector-based retrieval over unstructured text or reasoning over knowledge graphs, without providing a unified representation that combines both paradigms. Moreover, current benchmarks rarely provide ground-truth entities, relations, and fact-grounded question-answer pairs aligned with the underlying corpus. To address this gap, we introduce All Relations Lead to Rome (ARLtR), a unified framework for automated knowledge graph construction and fact-grounded question-answer generation. ARLtR jointly constructs a knowledge graph, embeddings, and question-answer pairs that are explicitly grounded in extracted entities, relations, and supporting textual evidence. We further instantiate the framework as a historical dataset centered on the Roman Empire, comprising over 19,000 entities, 16,000 chunks, and 8,400 question-answer pairs (https://huggingface.co/datasets/FaynePro/all-relations-lead-to-rome). By tightly coupling symbolic graph representations with dense retrieval representations, ARLtR facilitates the evaluation and development of hybrid retrieval systems and semantic steering approaches within a single coherent resource.