RealChart2Code: Advancing Chart-to-Code Generation with Real Data and Multi-Task Evaluation
For researchers evaluating VLM capabilities in code generation, this benchmark reveals a substantial gap in handling real-world, multi-panel visualizations, highlighting current limitations.
The paper introduces RealChart2Code, a benchmark with over 2,800 instances for evaluating chart-to-code generation from real-world data, and finds that leading VLMs struggle with complex, multi-panel charts, showing significant performance degradation compared to simpler benchmarks.
Vision-Language Models (VLMs) have demonstrated impressive capabilities in code generation across various domains. However, their ability to replicate complex, multi-panel visualizations from real-world data remains largely unassessed. To address this gap, we introduce \textbf{\texttt{RealChart2Code}}, a new large-scale benchmark with over 2,800 instances grounded in authentic datasets and featuring tasks with clear analytical intent. Crucially, it is the first benchmark to systematically evaluate chart generation from large-scale raw data and assess iterative code refinement in a multi-turn conversational setting. Our comprehensive evaluation of 14 leading VLMs on \texttt{RealChart2Code} reveals significant performance degradation compared to simpler benchmarks, highlighting their struggles with complex plot structures and authentic data. Our analysis uncovers a substantial performance gap between proprietary and open-weight models and confirms that even state-of-the-art VLMs often fail to accurately replicate intricate, multi-panel charts. These findings provide valuable insights into the current limitations of VLMs and guide future research directions. We release the benchmark and code at \url{https://github.com/Speakn0w/RealChart2Code}.