KcMF: A Knowledge-compliant Framework for Schema and Entity Matching with Fine-tuning-free LLMs
This addresses data integration challenges for users needing accurate matching in databases, though it is incremental as it builds on existing LLM methods.
The study tackled the problem of hallucinations and confusion in schema and entity matching tasks using large language models (LLMs) by proposing KcMF, a framework that enhances LLM performance without fine-tuning, resulting in an average F1-score improvement of 17.93% over non-LLM competitors.
Schema matching (SM) and entity matching (EM) tasks are crucial for data integration. While large language models (LLMs) have shown promising results in these tasks, they suffer from hallucinations and confusion about task instructions. This study presents the Knowledge-Compliant Matching Framework (KcMF), an LLM-based approach that addresses these issues without the need for domain-specific fine-tuning. KcMF employs a once-and-for-all pseudo-code-based task decomposition strategy to adopt natural language statements that guide LLM reasoning and reduce confusion across various task types. We also propose two mechanisms, Dataset as Knowledge (DaK) and Example as Knowledge (EaK), to build domain knowledge sets when unstructured domain knowledge is lacking. Moreover, we introduce a result-ensemble strategy to leverage multiple knowledge sources and suppress badly formatted outputs. Extensive evaluations confirm that KcMF clearly enhances five LLM backbones in both SM and EM tasks while outperforming the non-LLM competitors by an average F1-score of 17.93%.