Efficient Multi-Agent Collaboration with Tool Use for Online Planning in Complex Table Question Answering
This addresses the problem of high costs and accessibility in table question answering for researchers and practitioners by offering an efficient, reproducible alternative.
The paper tackles complex table question answering by introducing Multi-Agent Collaboration with Tool use (MACT), a framework that avoids closed-source models and fine-tuning, achieving state-of-the-art performance on three out of four benchmarks and matching GPT-4 on two benchmarks.
Complex table question answering (TQA) aims to answer questions that require complex reasoning, such as multi-step or multi-category reasoning, over data represented in tabular form. Previous approaches demonstrated notable performance by leveraging either closed-source large language models (LLMs) or fine-tuned open-weight LLMs. However, fine-tuning LLMs requires high-quality training data, which is costly to obtain, and utilizing closed-source LLMs poses accessibility challenges and leads to reproducibility issues. In this paper, we propose Multi-Agent Collaboration with Tool use (MACT), a framework that requires neither closed-source models nor fine-tuning. In MACT, a planning agent and a coding agent that also make use of tools collaborate to answer questions. Our experiments on four TQA benchmarks show that MACT outperforms previous SoTA systems on three out of four benchmarks and that it performs comparably to the larger and more expensive closed-source model GPT-4 on two benchmarks, even when using only open-weight models without any fine-tuning. We conduct extensive analyses to prove the effectiveness of MACT's multi-agent collaboration in TQA.