ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning
This addresses the problem of limited datasets for complex reasoning in chart question answering, providing a new benchmark for researchers in AI and data visualization.
The authors introduced ChartQA, a large-scale benchmark with 9.6K human-written and 23.1K generated questions for complex visual and logical reasoning over charts, and developed transformer-based models that achieved state-of-the-art results on this and previous datasets.
Charts are very popular for analyzing data. When exploring charts, people often ask a variety of complex reasoning questions that involve several logical and arithmetic operations. They also commonly refer to visual features of a chart in their questions. However, most existing datasets do not focus on such complex reasoning questions as their questions are template-based and answers come from a fixed-vocabulary. In this work, we present a large-scale benchmark covering 9.6K human-written questions as well as 23.1K questions generated from human-written chart summaries. To address the unique challenges in our benchmark involving visual and logical reasoning over charts, we present two transformer-based models that combine visual features and the data table of the chart in a unified way to answer questions. While our models achieve the state-of-the-art results on the previous datasets as well as on our benchmark, the evaluation also reveals several challenges in answering complex reasoning questions.