POLYCHARTQA: Benchmarking Large Vision-Language Models with Multilingual Chart Question Answering
This addresses the problem of limited accessibility for global audiences in chart understanding by providing a foundational benchmark for advancing inclusive vision-language models, though it is incremental as it extends existing English-centric benchmarks to multilingual settings.
The authors tackled the lack of multilingual benchmarks for chart understanding by introducing PolyChartQA, a large-scale dataset with 22,606 charts and 26,151 QA pairs across 10 languages, revealing a significant performance gap between English and other languages in vision-language models.
Charts are a universally adopted medium for interpreting and communicating data. However, existing chart understanding benchmarks are predominantly English-centric, limiting their accessibility and applicability to global audiences. In this paper, we present PolyChartQA, the first large-scale multilingual chart question answering benchmark covering 22,606 charts and 26,151 question-answering pairs across 10 diverse languages. PolyChartQA is built using a decoupled pipeline that separates chart data from rendering code, allowing multilingual charts to be flexibly generated by simply translating the data and reusing the code. We leverage state-of-the-art LLM-based translation and enforce rigorous quality control in the pipeline to ensure the linguistic and semantic consistency of the generated multilingual charts. PolyChartQA facilitates systematic evaluation of multilingual chart understanding. Experiments on both open- and closed-source large vision-language models reveal a significant performance gap between English and other languages, especially low-resource ones with non-Latin scripts. This benchmark lays a foundation for advancing globally inclusive vision-language models.