ChartAttack: Testing the Vulnerability of LLMs to Malicious Prompting in Chart Generation
This addresses a security risk for users of MLLM-based chart generation systems, highlighting an urgent need for robustness, but it is incremental as it focuses on a specific vulnerability rather than a new paradigm.
The paper tackles the problem of multimodal large language models (MLLMs) being vulnerable to malicious prompting in chart generation, introducing ChartAttack to generate misleading charts that reduce QA accuracy by an average of 19.6 points in in-domain settings and 20.2 points in human studies.
Multimodal large language models (MLLMs) are increasingly used to automate chart generation from data tables, enabling efficient data analysis and reporting but also introducing new misuse risks. In this work, we introduce ChartAttack, a novel framework for evaluating how MLLMs can be misused to generate misleading charts at scale. ChartAttack injects misleaders into chart designs, aiming to induce incorrect interpretations of the underlying data. Furthermore, we create AttackViz, a chart question-answering (QA) dataset where each (chart specification, QA) pair is labeled with effective misleaders and their induced incorrect answers. Experiments in in-domain and cross-domain settings show that ChartAttack significantly degrades the QA performance of MLLM readers, reducing accuracy by an average of 19.6 points and 14.9 points, respectively. A human study further shows an average 20.2 point drop in accuracy for participants exposed to misleading charts generated by ChartAttack. Our findings highlight an urgent need for robustness and security considerations in the design, evaluation, and deployment of MLLM-based chart generation systems. We make our code and data publicly available.