MRT at SemEval-2025 Task 8: Maximizing Recovery from Tables with Multiple Steps
This addresses the problem of improving accuracy in table-based QA for NLP researchers, but it is incremental as it builds on existing LLM methods with optimized prompts.
The paper tackled the SemEval 2025 Task 8 challenge for question-answering over tabular data by using a multi-step approach with Python code generation via LLMs, achieving a score of 70.50% for subtask 1.
In this paper we expose our approach to solve the \textit{SemEval 2025 Task 8: Question-Answering over Tabular Data} challenge. Our strategy leverages Python code generation with LLMs to interact with the table and get the answer to the questions. The process is composed of multiple steps: understanding the content of the table, generating natural language instructions in the form of steps to follow in order to get the answer, translating these instructions to code, running it and handling potential errors or exceptions. These steps use open source LLMs and fine grained optimized prompts for each task (step). With this approach, we achieved a score of $70.50\%$ for subtask 1.