RACOON: An LLM-based Framework for Retrieval-Augmented Column Type Annotation with a Knowledge Graph
This work addresses data exploration and integration challenges for users handling tabular data, representing an incremental improvement over existing LLM-based methods.
The paper tackles the problem of column type annotation by augmenting large language models with knowledge graph context, achieving up to a 0.21 micro F-1 improvement over vanilla LLM inference.
As an important component of data exploration and integration, Column Type Annotation (CTA) aims to label columns of a table with one or more semantic types. With the recent development of Large Language Models (LLMs), researchers have started to explore the possibility of using LLMs for CTA, leveraging their strong zero-shot capabilities. In this paper, we build on this promising work and improve on LLM-based methods for CTA by showing how to use a Knowledge Graph (KG) to augment the context information provided to the LLM. Our approach, called RACOON, combines both pre-trained parametric and non-parametric knowledge during generation to improve LLMs' performance on CTA. Our experiments show that RACOON achieves up to a 0.21 micro F-1 improvement compared against vanilla LLM inference.