Pandora: A Code-Driven Large Language Model Agent for Unified Reasoning Across Diverse Structured Knowledge
This addresses the challenge of leveraging knowledge transfer between different structured knowledge reasoning tasks for researchers and practitioners in AI and data science, representing a novel method rather than an incremental improvement.
The paper tackles the problem of Unified Structured Knowledge Reasoning (USKR) by proposing Pandora, a framework that uses Python's Pandas API to align with LLM pre-training and generate executable code for answering natural language questions across diverse structured sources, achieving performance that outperforms existing unified frameworks and competes with task-specific methods on four benchmarks.
Unified Structured Knowledge Reasoning (USKR) aims to answer natural language questions (NLQs) by using structured sources such as tables, databases, and knowledge graphs in a unified way. Existing USKR methods either rely on employing task-specific strategies or custom-defined representations, which struggle to leverage the knowledge transfer between different SKR tasks or align with the prior of LLMs, thereby limiting their performance. This paper proposes a novel USKR framework named \textsc{Pandora}, which takes advantage of \textsc{Python}'s \textsc{Pandas} API to construct a unified knowledge representation for alignment with LLM pre-training. It employs an LLM to generate textual reasoning steps and executable Python code for each question. Demonstrations are drawn from a memory of training examples that cover various SKR tasks, facilitating knowledge transfer. Extensive experiments on four benchmarks involving three SKR tasks demonstrate that \textsc{Pandora} outperforms existing unified frameworks and competes effectively with task-specific methods.