CABINET: Content Relevance based Noise Reduction for Table Question Answering
This addresses the issue of noise in table question-answering for users relying on large language models, offering an incremental improvement by enhancing existing methods with relevance-based filtering.
The paper tackles the problem of noise from irrelevant parts of tables in question-answering tasks, which distracts large language models and reduces performance, by proposing CABINET, a framework that uses an unsupervised relevance scorer and weakly supervised parsing to focus on relevant data, resulting in significant outperformance over baselines and new state-of-the-art results on datasets like WikiTQ, FeTaQA, and WikiSQL.
Table understanding capability of Large Language Models (LLMs) has been extensively studied through the task of question-answering (QA) over tables. Typically, only a small part of the whole table is relevant to derive the answer for a given question. The irrelevant parts act as noise and are distracting information, resulting in sub-optimal performance due to the vulnerability of LLMs to noise. To mitigate this, we propose CABINET (Content RelevAnce-Based NoIse ReductioN for TablE QuesTion-Answering) - a framework to enable LLMs to focus on relevant tabular data by suppressing extraneous information. CABINET comprises an Unsupervised Relevance Scorer (URS), trained differentially with the QA LLM, that weighs the table content based on its relevance to the input question before feeding it to the question-answering LLM (QA LLM). To further aid the relevance scorer, CABINET employs a weakly supervised module that generates a parsing statement describing the criteria of rows and columns relevant to the question and highlights the content of corresponding table cells. CABINET significantly outperforms various tabular LLM baselines, as well as GPT3-based in-context learning methods, is more robust to noise, maintains outperformance on tables of varying sizes, and establishes new SoTA performance on WikiTQ, FeTaQA, and WikiSQL datasets. We release our code and datasets at https://github.com/Sohanpatnaik106/CABINET_QA.