ExpliCIT-QA: Explainable Code-Based Image Table Question Answering
This addresses the explainability gap in table visual question answering for sensitive domains like finance and healthcare where auditing results is critical, though it is incremental as it extends a previous approach.
The paper tackled the problem of making table question answering from images more explainable by developing ExpliCIT-QA, a multimodal pipeline that generates step-by-step natural language explanations and executable code, resulting in improved interpretability and transparency on the TableVQA-Bench benchmark.
We present ExpliCIT-QA, a system that extends our previous MRT approach for tabular question answering into a multimodal pipeline capable of handling complex table images and providing explainable answers. ExpliCIT-QA follows a modular design, consisting of: (1) Multimodal Table Understanding, which uses a Chain-of-Thought approach to extract and transform content from table images; (2) Language-based Reasoning, where a step-by-step explanation in natural language is generated to solve the problem; (3) Automatic Code Generation, where Python/Pandas scripts are created based on the reasoning steps, with feedback for handling errors; (4) Code Execution to compute the final answer; and (5) Natural Language Explanation that describes how the answer was computed. The system is built for transparency and auditability: all intermediate outputs, parsed tables, reasoning steps, generated code, and final answers are available for inspection. This strategy works towards closing the explainability gap in end-to-end TableVQA systems. We evaluated ExpliCIT-QA on the TableVQA-Bench benchmark, comparing it with existing baselines. We demonstrated improvements in interpretability and transparency, which open the door for applications in sensitive domains like finance and healthcare where auditing results are critical.