Pandora: Leveraging Code-driven Knowledge Transfer for Unified Structured Knowledge Reasoning
This addresses the challenge of integrating diverse structured knowledge sources for natural language question answering, offering a more cohesive approach that could benefit AI systems in domains like data analysis and knowledge management, though it appears incremental by building on existing LLM pre-training.
The paper tackles the problem of Unified Structured Knowledge Reasoning (USKR) by introducing Pandora, a framework that uses a code-based representation and knowledge transfer to improve cross-task performance, achieving superior results on six benchmarks across three tasks compared to existing unified methods and competitive performance with task-specific ones.
Unified Structured Knowledge Reasoning (USKR) aims to answer natural language questions by using structured sources such as tables, databases, and knowledge graphs in a unified way. Existing USKR methods rely on task-specific strategies or bespoke representations, which hinder their ability to dismantle barriers between different SKR tasks, thereby constraining their overall performance in cross-task scenarios. In this paper, we introduce \textsc{Pandora}, a novel USKR framework that addresses the limitations of existing methods by leveraging two key innovations. First, we propose a code-based unified knowledge representation using \textsc{Python}'s \textsc{Pandas} API, which aligns seamlessly with the pre-training of LLMs. This representation facilitates a cohesive approach to handling different structured knowledge sources. Building on this foundation, we employ knowledge transfer to bolster the unified reasoning process of LLMs by automatically building cross-task memory. By adaptively correcting reasoning using feedback from code execution, \textsc{Pandora} showcases impressive unified reasoning capabilities. Extensive experiments on six widely used benchmarks across three SKR tasks demonstrate that \textsc{Pandora} outperforms existing unified reasoning frameworks and competes effectively with task-specific methods.