Enhancing Question Answering on Charts Through Effective Pre-training Tasks
This addresses the problem of chart understanding for VisualQA applications, but it is incremental as it builds on existing models with modest gains.
The paper tackled the limitation of VisualQA models on charts by identifying their underperformance in structural, visual, and numerical questions through behavioral analysis, and proposed three pre-training tasks that improved the baseline model by an average of 1.7% on three chart datasets.
To completely understand a document, the use of textual information is not enough. Understanding visual cues, such as layouts and charts, is also required. While the current state-of-the-art approaches for document understanding (both OCR-based and OCR-free) work well, a thorough analysis of their capabilities and limitations has not yet been performed. Therefore, in this work, we addresses the limitation of current VisualQA models when applied to charts and plots. To investigate shortcomings of the state-of-the-art models, we conduct a comprehensive behavioral analysis, using ChartQA as a case study. Our findings indicate that existing models particularly underperform in answering questions related to the chart's structural and visual context, as well as numerical information. To address these issues, we propose three simple pre-training tasks that enforce the existing model in terms of both structural-visual knowledge, as well as its understanding of numerical questions. We evaluate our pre-trained model (called MatCha-v2) on three chart datasets - both extractive and abstractive question datasets - and observe that it achieves an average improvement of 1.7% over the baseline model.