ProTrix: Building Models for Planning and Reasoning over Tables with Sentence Context
This work addresses the challenge of improving planning and reasoning for tabular tasks, offering generalizability and interpretability, but it is incremental as it builds on existing prompting and fine-tuning methods.
The paper tackles the problem of answering diverse user queries over tables with sentence context by proposing a Plan-then-Reason framework, which enhances table reasoning abilities and leads to models like ProTrix that generalize to unseen tasks with only 6k training instances and outperform baselines using fewer API calls.
Tables play a crucial role in conveying information in various domains. We propose a Plan-then-Reason framework to answer different types of user queries over tables with sentence context. The framework first plans the reasoning paths over the context, then assigns each step to program-based or textual reasoning to reach the final answer. This framework enhances the table reasoning abilities for both in-context learning and fine-tuning methods. GPT-3.5-Turbo following Plan-then-Reason framework surpasses other prompting baselines without self-consistency while using less API calls and in-context demonstrations. We also construct an instruction tuning set TrixInstruct to evaluate the effectiveness of fine-tuning with this framework. We present ProTrix model family by finetuning models on TrixInstruct. Our experiments show that ProTrix family generalizes to diverse unseen tabular tasks with only 6k training instances. We further demonstrate that ProTrix can generate accurate and faithful explanations to answer complex free-form questions. Our work underscores the importance of the planning and reasoning abilities towards a model over tabular tasks with generalizability and interpretability. We open-source our dataset and models at https://github.com/WilliamZR/ProTrix.