From Charts to Code: A Hierarchical Benchmark for Multimodal Models

[Project Page]https://csu-jpg.github.io/Chart2Code.github.io \checkdata[Code]https://github.com/CSU-JPG/Chart2Code \checkdata[Data]https://huggingface.co/datasets/CSU-JPG/Chart2Code

We introduce Chart2Code, a new benchmark for evaluating the natural language to chart code generation capabilities of large multimodal models (LMMs). Chart2Code is explicitly designed from a user-driven perspective, capturing diverse real-world scenarios and progressively increasing task difficulty. It consists of three levels: Level 1 (Chart Reproduction) reproduces charts from a reference figure and user query; Level 2 (Chart Editing) involves complex modifications such as changing chart types or adding elements; and Level 3 (Long-Table to Chart Generation) requires models to transform long, unprocessed tables into faithful charts following user instructions. To our knowledge, this is the first hierarchical benchmark that reflects practical chart2code usage while systematically scaling task complexity. In total, Chart2Code contains 2,186 tasks across 22 chart types, paired with multi-level evaluation metrics that assess both code correctness and the visual fidelity of rendered charts. We benchmark 29 state-of-the-art (SoTA) LMMs, including both proprietary and the latest open-source models such as GPT-5.2, Qwen3-VL, InternVL3/3.5, MiMo-VL, and Seed-1.6-VL. Experimental results demonstrate that even the SoTA model GPT-5.2 averages 72.21 on code-based evaluation and only 33.41 on chart-quality assessment across the editing tasks, underscoring the difficulty of Chart2Code. We anticipate this benchmark will drive advances in multimodal reasoning and foster the development of more robust and general-purpose LMMs. The code and data are available on Chart2Code

1 Introduction

Charts are one of the most powerful tools in scientific publications and business reports, they distill large amounts of structured data into clear and persuasive visuals. With the rapid progress of large multimodal models (LMMs) [21, 1], it becomes increasingly realistic to envision AI systems that not only interpret visual charts [29] but also generate executable plotting code, a task we refer to as chart-to-code (chart2code). Such capabilities can significantly enhance productivity by automating visualization creation, enabling reproducibility.

However, exisiting chart to code benchmarks exsist the performance saturation issues such as ChartMimic [35], Plot2Code [30]. Previous models GPT-4o already reached 82.2% scores in ChartMimic. With the rapid advancement of frontier models such as GPT-5.2 [22], ChartMimic can no longer serve as a sufficiently challenging or discriminative benchmark for evaluating current or future models.

Additionally, there is a lack of a systematic chart-to-code benchmark that covers more comprehensive and realistic human–AI interactive chart-coding scenarios. Users rarely stop at simple chart reproduction that ChartMimic does, they need to edit figures to improve their work productivity avoiding boring repetitive works. For non-expert users who are not data analyst, they often work with long and raw tables that must be distilled into interpretable plots. However, when applied to these more common and demanding scenarios, the exisiting state-of-the-art (SOTA) models often struggle, revealing substantial gaps in their practical ability (refer to Appendix 9 for examples). This discrepancy creates a mismatch between reported benchmark performance and real-world utility, highlighting the need for a challenging and relistic benchmark that more comprehensively reflects everyday chart2code challenges.

Figure 1: Chart2Code covers three progressively challenging levels: reproduction, editing, and long-table to chart generation. It provides a user-driven and diverse benchmark that better reflects real-world chart2code demands.

Motivated by these observations, we introduce Chart2Code (Figure 1), a new benchmark designed to rigorously evaluate chart generation capabilities of LMMs under progressively challenging conditions. Chart2Code consists of three difficulty levels: Level 1 (Chart Reproduction) targets mimicking a reference figure and instruction; Level 2 (Chart Editing) requires complex and precise editing, such as changing chart types or adding new elements; Level 3 (Long-Table to Chart Generation) presents the most demanding setting, where models must convert long, unprocessed data tables into faithful charts from user instructions. This hierarchical design reflects real-world usage while progressively increasing difficulty, and its distinctions from prior benchmarks are highlighted in Table 1.

We comprehensively benchmark 29 state-of-the-art LMMs, including both proprietary and open-weight models, across the three levels of Chart2Code. Our results show that while LMMs demonstrate promising capabilities on simple reproduction tasks, their performance deteriorates sharply on complex editing and long-context data-to-chart generation. Together, these findings reveal the unsolved challenges of chart2code generation and point to future directions for building more reliable visualization assistants.

We present Chart2Code, the first hierarchical benchmark targeting chart2code generation with progressively more challenging tasks.

We propose multi-level evaluation protocols that jointly assess code executability and visual fidelity, offering a comprehensive lens on model performance.

We provide an extensive empirical study across 29 mainstream LMMs, yielding new insights into their strengths, weaknesses, and design trade-offs for chart generation.

Table 1: Comparison of existing chart-to-code benchmarks. Ref. Fig.: Reference Figure; Instr.: Instruction; Text Data: Text-format data; Fig. Data: Figure-format data; NL: Natural language.

Benchmark Input Type Task Category Output Metric Datasets Size

Ref. Fig. Instr. Text Data Fig. Data Chart Reproduction Chart Editing Long-table to Chart Rule-based GPT-score

CharXiv [29] ✗ ✓ ✗ ✗ ✗ ✗ ✗ NL ✓ ✗ 2323

Plot2Code [30] ✓ ✓ ✗ ✗ ✓ ✗ ✗ Code ✗ ✓ 132

AcademiaChart [38] ✓ ✓ ✗ ✗ ✓ ✗ ✗ Code ✓ ✓ 2525

Chartmimic [35] ✓ ✓ ✗ ✗ ✓ ✗ ✗ Code ✓ ✓ 4800

ChartEdit [40] ✓ ✓ ✗ ✗ ✗ ✓ ✗ Code ✗ ✓ 233

Chart2Code (Ours) ✓ ✓ ✓ ✓ ✓ ✓ ✓ Code ✓ ✓ 2186

2 Related Work

Large Multimodal Models.

Thanks to the success of proprietary LMMs such as GPT-5 [22], Gemini-3-Pro [8], and Claude-Sonnet-4 [1], we see the dawn of building AI agents for addressing realistic applications. In the academic community, we see enormous excellent open-source models: MiMo-VL [31], QwenVL-series [3, 27], and InternVL-series [28, 42], MolMo [6], Kimi-VL [25] LLaVA-series [12, 17, 13], Deepseek-VL [18], and GLM-4V [7].

Agentic Benchmarks.

The rapid progress of foundation LLMs and LMMs has motivated the creation of diverse agentic benchmarks, spanning GUI automation [32, 39, 16, 11], agentic coding [10, 36], tool use [37], AI research assistance [20], and chart reasoning [29]. We focus on chart2code, a practical task central to everyday workflows for researchers and professionals. Despite progress, even the best proprietary LMMs still fail to generate faithful charts from long, raw tables, underscoring the need for future modeling advances.

Chart Understanding to Code Generation.

Chart understanding has evolved through a series of benchmarks that progressively expand task complexity. ChartQA [19] first established large-scale visual question answering over charts, combining logical and visual reasoning. ChartXiv [29] extended this line by introducing scientific charts with expert-designed questions, further exposing the gap between multimodal models and human performance.

Moving beyond QA, Chart2Code benchmarks address faithful chart generation. ChartMimic [35] formalized this by requiring code synthesis from chart images and instructions, while ChartEdit[40] and ChartM3[33] emphasize interactive modification, where models edit chart-rendering code following natural-language instructions.

Extending chart generation, StarVector [24] proposes a vision-language approach to produce scalable vector graphics from visual or textual inputs. FigEdit[14] highlights the gap between chart editing and general figure editing. ChartBench [34] focuses on chart question answering (VQA), related to CharXiv [29], while BigDocs [23] targets multimodal document understanding with tasks such as Screen2HTML, Image2SVG, and Image2Flow. ChartGalaxy [15] introduces a large-scale dataset for infographic chart generation, focusing on complex visual layouts and design variations. However, none of these works directly address the chart-to-code formulation studied in this paper.

Although GPT-4o achieves high scores on ChartMimic (83.2) and ChartEdit (93.6), it still struggles with realistic chart2code tasks, motivating a more challenging benchmark for reliable evaluation.

3 Chart2Code: From Visual Charts to Code

3.1 Task Definition of Chart2Code

The Chart2Code can be formulated as: C=f⁡(R,I,D) where, R is the visual reference chart (e.g., chart screenshot), I is the user’s natural language instruction and C is the executable Python code generated by LMM (f). D represents the optional underlying data source provided to ground the generation. To accommodate real-world scenarios, we design D to support three modalities: raw text, visual data tables (e.g., screenshots of spreadsheets), and structured files (e.g., Excel). To ensure rigor and comprehensiveness, we designed three tasks of increasing difficulty.

3.2 Hierarchical task design

Level 1 (Chart Reproduction):

This foundational level assesses the model’s fidelity in visual mimicry and is divided into two scenarios.

Visual-Only Reproduction (R+I,D=∅): The LMM must generate code to reproduce the visual appearance and underlying data of the reference chart R purely from the image. We denote this setting as DR (Direct Reproduction). This setting primarily evaluates the model’s visual perception and understanding capabilities.

Style Transfer with Data (R+D+I): The LMM is provided with an external data source D and must generate code to visualize it following the style and layout of R. We define two variants: CRD(Customize Raw Data), where D is raw tabular data, and CFD(Customize Figure Data), where D is structured figure-level data. This setting reflects real-world workflows in which users aim to replicate chart styles given available data [35, 30, 38].

Level 2 (Chart Editing):

At this level, LMM edits the reference chart (R) as instructed, performing basic operations such as style changes, type swaps, and color adjustments. These tasks, similar to the editing functions in [40], primarily test the LMM’s fundamental chart understanding ability. Going a step further, Level 2 involves more sophisticated editing operations. These include calculating correlation coefficients based on chart data, plotting trend lines, generating data analysis tables, splitting charts by data category, incorporating new data analyses and so on. Beyond mere understanding, these tasks further stress-test the LMM’s inferential reasoning. This setting reflects a sophisticated real-world scenario where users demand extended analytical insights and complex visual synthesis, rather than the simplistic localized edits addressed in previous benchmarks like [40].

Level 3 (Long-Table to Chart):

The final level of our benchmark is also the hardest task setting. It reflects the most common usage in real-world chart creating scenarior. To summarize, the challenges of Level 3 requires the model to (1) absorb long-context inputs, (2) perform reasoning and information retrieval, (3) conduct mathematical calculation, (4) generate executable code by following the diverse user queries. So the Level 3 is distinct from Level 1 in task definition. This task also reflects the scenario that a non-expert user uses the LLM/LMM for data visualization with raw tabular data. This is also a domain that has not been addressed in previous works[35, 40, 19].

Figure 2: Collected charts distribution.

Figure 3: Detailed data statistic.

Statistic Number

GT Charts

Total charts 2,186

- Level 1 / 2 / 3 charts 863 / 1,010 / 313

Unique charts 967

- Unique Level 1 / 2 / 3 charts 719 / 0 / 248

Instructions

Total instructions 2,222

- Level 1 / 2 / 3 instructions 899 / 1,010 / 313

Unique instructions 1,468

- Unique instructions - Level 1 / 2 / 3 145 / 1,010 / 313

Maximum instruction length - Level 1/ 2 / 3 522 / 578 / 390

Average instruction length - Level 1 / 2 / 3 140.5 / 267.3 / 120.8

GT Code (Lengths/Lines)

Maximum code length - Level 1 / 2 / 3 96,563 / 1,1860 / 790,130

Average code length - Level 1 / 2 / 3 2,714.98 / 3,063.2 / 11,754.3

Maximum code lines - Level 1 / 2 / 3 842 / 342 / 388

Average code lines - Level 1 / 2 / 3 71.6 / 85.0 / 69.5

Extremely Long-Table Data

Total Excel files 71

Average lines per file 2,647

Maximum lines 30,427

Average data entries 810,329.3

Maximum data entries 72,705

3.3 Data Statistics and Analysis

Chart2Code comprises 2,186 tasks across three levels–863/1,010/313 for L1/L2/L3–spanning 22/19/12 chart families (e.g., radar, heatmap, scatter, box, tree, error-bar, pie, violin; see Fig. 3). To maximize diversity, Level 1 emphasizes unique charts (719 unique). Level 2 reuses Level 1 charts with at least one edit instruction per chart, resulting in 1,010 unique, non-duplicated edits. Level 3 (LT2Chart) includes 245 charts and 313 instructions derived from web-sourced long tables, making annotation and ground-truth code especially challenging. As summarized in Tab. 3, the instruction scale and substantial code lengths highlight the breadth and difficulty of Chart2Code. The data source and annotation details are provided in Appendix 11.1

3.4 Evaluation

3.4.1 Overall

To comprehensively evaluate model performance on this benchmark, we first establish the Code executable rate, Exec.Rate as the primary metric. This metric directly measures the model’s capability to generate executable code, with its definition and calculation detailed in Appendix 15.1. Furthermore, we introduce a multi-level and multi-dimensional evaluation framework to evaluate models at both the code-level and chart-level. At the code-level, we propose rule-based and LLM-based evaluation frameworks to compare the similarity in visual outcomes between the ground-truth and generated code. At the chart-level, we employ LMMs to evaluate the visual fidelity of the predicted charts.

3.4.2 Code-Level

Base Evaluation Metrics.

To strictly evaluate visual fidelity at the code level, we propose a rule-based base evaluation method, referred to as Base. Leveraging insights into commonly used scientific plotting libraries—such as Matplotlib, Seaborn, Wordcloud, and Networkx—we parse the underlying plotting object Figure. We isolate eight critical dimensions that are fundamental to the evaluation of visual fidelity (detailed in Appendix 15.2).

Compared to the four-dimension heuristic evaluation in [35], which relies on function trackers and code injection, our method offers broader dimensional coverage, stronger applicability, faster execution, and higher accuracy (detailed comparison in Appendix 15.2).

LLM Evaluation Metrics.

Recognizing that rule-based matching remains suboptimal for comprehensively evaluating visual fidelity at the code level (see Appendix 1), we leverage GPT-5-mini to evaluate the code, referred to as LLM, thereby exploring its potential to enhance code-level visual fidelity evaluation. By employing more stringent prompts, we evaluate the similarity across various chart dimensions from a code-level perspective and further analyze the performance gap between the code-level and chart-level assessments (see Appendix 12.1).

3.4.3 Chart-Level

LMM Evaluation Metrics.

Due to the inherent discrepancy between code-level and chart-level representations, evaluating visual fidelity solely from a code-based perspective is insufficient. Following prior work [35, 40], we utilize GPT-5-mini to perform multi-dimensional similarity scoring between ground-truth and generated charts, referred to as LMM.

Human Evaluation.

Given that current LMMs exhibit certain biases in chart visual perception (see Appendix 9), we also designed a human evaluation system to analyze the consistency between LMM and human assessments. we conducted a human evaluation study involving 20 undergraduate students from the School of Computer Science. Participants assessed both the ground truth (GT) charts and the generated charts across multiple dimensions, aiming for a comprehensive evaluation of the reproduction quality. The results demonstrate a strong correlation between human scores and LMM scores (see Appendix 14).

4 Experiments

4.1 Experiments Setup

Models. We conducted tests on 29 widely-used open-source models and proprietary models to evaluate their performance on our benchmark. For the open-source models, we selected 12 representative vision-language models, with total parameters ranging from 7B to 72B, including: Qwen2-VL (7B, 72B), Qwen2.5-VL (7B, 72B), Deepseek-VL (7B), Kimi-VL (7B), MiMo-VL-SFT (7B), MiMo-VL-RL (7B), InternVL-2.5 (8B, 38B), InternVL-3 (8B, 38B), InternVL-3.5 (8B, 38B), GLM-4V (9B), LLAVA-onevision-si (7B), LLAVA-onevision-ov (7B), Molmo (7B),Qwen3-VL. For proprietary models, we selected the five most popular multimodal large models, including: Gemini-3-pro, Claude-sonnet-4, GPT-5.2, Seed-1.5-VL, and Seed-1.6-VL.

Configuration. All experiments were conducted on NVIDIA V100 GPUs. Models in the 7B–9B range were evaluated using two GPUs with model parallelism; detailed per-level configurations are provided in Tab. 19. Empirically, non-thinking models required only 4,096 tokens for inference, with negligible truncation except for InternVL-3.5-38B and the Qwen3-VL series. To preserve visual fidelity, images were fed at their native resolution, and the maximum input pixel setting supported by each model was used to ensure complete processing of chart details.

Table 2: Evaluation results on Chart Reproduction (Level 1) with various LMMs. Each task includes a reference chart as input. DR: input without the any data. CRD: input with customized text-format table data. CFD: input with customized figure-format table data. Exec.Rate: execution rate(%); Base: the multi-dimensional weighted Base-Score. The Base score uses the same dimension-wise weighting scheme as the LLM-score(LLM). More detail Metrics in Appendix 12. We use GPT-5-mini as the base model for both LLM-score and LMM-score(LMM); Base, LLM, LMM: scores on a 0–100 scale (higher is better).

Model Direct Reproduction(DR) Customize Raw Data(CRD) Customize Figure Data(CFD)

Exec.Rate Base LLM LMM Exec.Rate Base LLM LMM Exec.Rate Base LLM LMM

Proprietary

Gemini-3-Pro 97.50 78.65 75.86 45.42 100.0 69.23 69.76 40.72 99.07 70.78 71.12 32.85

Claude-Sonnet-4 96.52 65.60 70.69 32.36 100.0 61.46 55.03 40.63 93.52 65.27 65.99 26.44

GPT-5.2 97.08 79.91 77.88 43.73 97.22 66.31 65.51 39.26 99.07 73.02 71.42 35.40

Seed-1.5-VL 87.34 63.85 53.84 26.40 97.22 65.76 58.73 34.74 79.63 65.58 64.19 19.53

Seed-1.6-VL 82.61 62.46 50.85 26.16 94.44 60.69 57.87 28.44 79.63 66.22 63.12 25.51

Open-Source LMMs (non-thinking)

LLaVA-OV-Qwen2-7B-SI 82.48 31.33 16.81 2.58 19.44 49.73 35.93 15.14 0.00 - - -

LLaVA-OV-Qwen2-7B-OV 72.60 34.33 54.85 3.42 8.33 32.68 15.67 6.00 0.00 - - -

DeepSeek-VL-7B 36.44 37.49 22.02 3.37 55.56 50.73 29.28 9.25 4.63 30.37 25.20 5.60

kimi-VL-A3B 70.79 52.76 40.36 14.15 66.67 50.29 46.85 33.04 64.81 53.62 45.36 16.41

Qwen2-VL-7B 62.45 40.99 28.74 7.02 77.78 52.21 44.50 19.61 31.48 51.02 41.03 8.65

Qwen2-VL-72B 74.97 50.74 38.27 12.75 86.11 56.02 46.89 28.23 64.81 59.66 56.41 15.81

InternVL-2.5-8B 67.73 42.76 27.49 7.80 69.44 54.16 42.80 25.88 37.04 51.58 38.81 12.85

InternVL-2.5-38B 85.67 51.97 39.83 15.18 86.11 55.74 47.98 24.71 77.78 59.86 55.51 22.88

InternVL-3-8B 29.49 46.58 55.68 9.20 36.11 53.47 31.38 6.23 12.96 50.73 35.61 3.14

InternVL-3-38B 85.26 53.57 42.85 16.68 86.11 58.17 51.10 34.00 36.11 60.17 60.42 18.56

GLM-4V-9B 54.24 32.50 23.12 4.80 75.00 50.35 42.91 19.70 41.67 31.64 12.84 1.00

GLM-4.6v-Flash 54.52 66.67 55.38 31.70 61.11 63.44 52.75 32.45 55.56 65.64 60.23 22.57

Intern-VL-3.5-8B 72.18 52.01 41.29 15.41 69.44 37.87 14.58 1.68 49.07 53.18 48.85 10.87

Intern-VL-3.5-38B 84.01 55.78 42.54 19.78 86.11 60.66 53.16 38.83 24.07 61.15 60.69 21.84

MiMo-VL-7B-RL 43.12 64.41 49.82 21.43 66.67 59.73 52.21 25.38 42.59 64.39 57.68 21.44

MiMo-VL-7B-SFT 51.74 63.99 49.07 22.61 75.00 53.34 37.80 22.74 32.41 64.68 54.74 22.83

Qwen2.5-VL-7B 69.26 48.20 35.46 10.54 83.33 55.42 46.72 20.14 44.44 50.30 41.78 7.81

Qwen2.5-VL-72B 63.84 59.05 47.95 20.97 94.44 58.12 50.97 27.53 47.22 63.95 61.37 24.48

Molmo-7B-D 4.31 25.26 11.48 2.42 2.78 70.30 40.00 4.00 1.85 45.58 24.00 0.00

Qwen3-VL-30B-A3B 72.18 63.09 50.84 15.03 69.44 55.84 52.26 23.28 74.07 63.02 60.23 23.56

Qwen3-VL-32B 76.91 63.10 54.87 18.44 77.78 60.49 52.64 36.79 80.56 66.64 65.27 25.84

Open-Source LMMs (thinking)

MiMo-VL-7B-RL 21.14 70.45 60.00 30.04 75.00 64.58 57.96 24.96 37.96 68.05 64.38 29.83

MiMo-VL-7B-SFT 41.03 65.49 54.33 16.95 77.78 61.33 53.89 29.93 37.96 64.04 56.04 23.29

Qwen3-VL-30B-A3B 76.08 62.37 55.48 23.83 61.11 64.78 57.63 35.00 80.56 66.57 62.86 26.34

Table 3: Evaluation results on Chart Editing (Level 2) with various LMMs.

Model Exec. Rate Code-Level Chart-Level

Color Grid Layout Legend Visual Data Text Type Base LLM LMM

Proprietary

Gemini-3-Pro 97.23 52.32 75.31 86.45 63.33 81.58 62.75 77.16 93.86 70.78 72.21 33.41

Claude-Sonnet-4 90.20 47.17 65.29 55.32 56.51 81.50 54.88 80.52 93.29 63.65 66.45 25.40

GPT-5.2 96.04 58.44 80.83 61.51 58.16 84.65 64.66 83.77 94.52 70.93 75.66 33.03

Seed-1.5-VL 60.20 44.39 69.37 52.06 52.80 79.85 52.02 77.21 92.66 61.67 65.45 18.30

Seed-1.6-VL 70.00 43.61 69.37 52.25 51.32 79.70 53.11 79.46 92.32 61.77 65.40 17.43

Open-Source LMMs (non-thinking)

LLaVA-OV-Qwen2-7B-SI 2.57 15.35 59.38 53.85 30.79 56.34 22.59 52.70 55.33 38.43 37.40 6.40

LLaVA-OV-Qwen2-7B-OV 1.68 15.37 63.48 38.24 64.56 50.15 18.13 48.81 58.43 39.07 35.44 7.35

DeepSeek-VL-7B 30.46 15.32 42.58 34.44 26.24 52.95 22.36 51.56 68.43 35.16 26.87 4.20

kimi-VL-A3B 50.69 26.22 64.50 40.68 39.93 66.08 34.05 64.97 83.03 47.96 44.43 9.01

Qwen2-VL-7B 24.55 18.72 57.60 41.87 32.73 58.65 25.41 55.07 76.38 41.06 34.82 5.27

Qwen2-VL-72B 57.23 27.82 64.07 42.86 40.59 69.53 36.83 64.35 84.76 49.55 39.88 10.02

InternVL-2.5-8B 24.06 23.86 62.25 41.21 36.70 65.03 32.72 62.46 81.14 46.20 41.14 7.48

InternVL-2.5-38B 30.30 30.40 68.25 50.34 48.97 72.92 37.38 70.78 87.96 53.48 52.90 10.67

InternVL-3-8B 4.55 23.22 62.68 57.03 31.34 66.79 29.34 58.63 84.49 46.41 38.02 5.48

InternVL-3-38B 69.80 35.87 68.36 48.66 49.30 75.86 45.33 72.28 90.53 56.74 56.98 13.32

GLM-4V-9B 10.50 19.71 61.79 48.11 34.54 59.98 26.53 57.80 65.80 42.05 34.78 3.87

GLM-4.6V-Flash 18.22 48.06 69.23 50.13 51.20 81.56 54.78 77.90 91.86 62.76 63.25 17.91

InternVL-3.5-8B 37.52 29.55 68.21 43.87 41.56 69.85 39.58 67.41 83.82 51.30 49.52 9.71

InternVL-3.5-38B 4.26 44.11 92.54 67.22 60.24 72.35 40.38 76.53 88.56 62.64 57.48 8.28

MiMo-VL-7B-RL 17.23 42.72 70.76 54.30 46.81 74.50 49.23 74.57 89.75 59.42 56.95 14.54

MiMo-VL-7B-SFT 30.40 37.36 67.37 48.40 47.24 75.98 46.53 70.80 88.04 56.56 53.97 15.23

Qwen2.5-VL-7B 31.98 26.87 67.22 45.41 38.75 65.34 37.21 63.68 83.65 49.22 44.35 9.44

Qwen2.5-VL-72B 74.55 41.32 67.44 52.16 50.04 77.74 48.65 74.72 91.12 59.31 49.02 16.69

Molmo-7B-D 0.59 16.53 44.44 41.67 50.00 49.70 29.14 41.57 54.44 37.32 29.92 15.50

Qwen3-VL-30B-A3B 38.91 40.07 70.02 51.69 50.06 76.81 48.01 75.52 92.39 59.26 45.77 17.46

Qwen3-32B 63.76 42.90 71.98 50.76 52.24 77.96 52.35 77.43 92.09 61.30 64.15 18.27

Open-Source LMMs (thinking)

MiMo-VL-7B-RL 26.14 42.47 69.05 48.47 45.96 80.76 50.41 74.80 90.56 59.53 61.04 17.81

MiMo-VL-7B-SFT 30.50 40.56 68.22 48.10 46.65 78.42 48.68 73.48 89.86 58.32 59.33 16.38

Qwen3-VL-30B-A3B 13.76 44.00 65.02 48.92 49.19 78.19 50.18 76.18 89.60 59.55 63.04 14.37

Table 4: Evaluation results on Long-Table to Chart task (Level 3) with various LMMs.

Model Exec. Rate Code-Level Chart-Level

Color Grid Layout Legend Visual Data Text Type Base LLM LMM

Proprietary

Gemini-3-Pro 30.03 56.29 83.71 92.55 68.39 84.00 62.26 80.45 96.60 74.28 77.90 35.97

Claude-Sonnet-4 46.65 35.43 68.97 84.08 61.77 74.44 46.73 67.77 89.95 61.13 62.04 16.60

GPT-5.2 20.13 34.46 82.14 80.87 52.64 78.34 43.56 76.09 93.81 61.99 68.40 16.29

Seed-1.5-VL 10.86 44.55 76.89 93.94 69.30 73.52 51.75 73.57 95.96 67.58 75.59 22.85

Seed-1.6-VL 26.52 44.26 82.28 95.48 74.73 75.42 52.62 77.82 86.47 68.6 76.35 25.77

4.2 Main Experimental Results

4.2.1 Level-wise Comparison of Models

Level 1.

As shown in Tab. 2, proprietary models lead across DR, CRD, CFD achieving high executability but only moderate visual fidelity—for example, Gemini-3-Pro reaches 97.5/100/99.1% ER on DR/CRD/CFD while LMM-Scores stay around 33.85–45.42. CRD is “easy to run” (e.g., Gemini, GPT and Qwen2.5-VL-72B at ≈100% ER) yet still low-fidelity (≈27.53–40.72), confirming execution ≠ fidelity. CFD is the hardest: top proprietary models keep ≥99% ER but LMM-Scores remain ≈32.85–35.40, and many open-source models drop sharply (some 0 ER). Larger open-source backbones (Qwen3-VL-32B/MOE/Dense, InternVL-2-38B) close part of the execution gap but not the fidelity gap. A notable outlier is Molmo-7B-D with DR Base-Score ≈70.3, which is primarily attributable to statistical bias caused by its extremely low ER.

Level 2.

The results are presented in Tab. 3. Proprietary models sustain ∼90% ER (Gemini 97.23, Claude 90.20, GPT-5.2 96.04) and excel on code-level subskills, especially Layout/Type ≈86.45–94.52—yet chart-level remains modest (∼17.43–33.41), evidencing a persistent gap between syntactic compliance and rendered-image fidelity. Strong open-source systems improve executability (e.g., Qwen2.5-VL-72B 74.55%), but chart-level still lags (16.69). Smaller backbones struggle (e.g., LLaVA-OV-Qwen2-7B variants ≤2.57% ER). “Thinking” helps procedure more than pixels: MiMo-VL-7B-RL ER improves 17.23→26.14, and MiMo-VL-7B-SFT chart-level nudges 15.23→16.38, but absolute fidelity remains low. Notably, the thinking variant of Qwen3-VL-30B-A3B suffers from a low execution rate due to extensive truncation issues, further highlighting the substantial challenges this task poses for open-source models.

Level 3.

Tab. 4 presented the results. Coverage is limited because the benchmark is very hard: only a couple of open-source models could even complete inference, and on the proprietary side, five models were run, but overall ER is still <50%, primarily due to long-context inputs exceeding the maximum input limits. Among those that ran, ER drops to 10–46% (e.g., Seed-1.5-VL 10.86%), while code-level stays strong (Base ≈61–74 LLM ≈68–78), indicating structurally plausible code under long context. However, chart-level fidelity collapses (Gemini 35.97, Claude 16.60, GPT-5.2 16.29; Seed-1.5/1.6-VL 22.85/25.77), showing that turning lengthy raw tables into pixel-accurate charts is the main bottleneck; this is mainly because current models are unable to simultaneously handle extremely long textual inputs and reference images, and extract the data processing and plotting requirements from the instructions.

4.2.2 Analysis

Execution vs. Complexity: From level 2 to Level 3, ER for proprietary systems drops from 90% in Tab. 3 to 10–30% on Level 3 (Gemini 30.03, Claude 46.65, GPT-5.2 20.13 in Tab. 4). This mirrors the jump in reasoning load (long-context/table parsing, multi-constraint edits), showing that being able to run code at level 2 does not translate to robust end-to-end success at Level 3. We concluded execution success declines steeply with task complexity, even for top proprietary models.

Code vs. Visual Fidelity: On level 2 (Tab. 3), proprietary models achieve moderate Base-Score and LLM-Score. (e.g., Gemini 70.78/72.21, Claude 63.65/66.45, GPT-5.2 70.93/75.66), yet chart-level GPT-Score is only 17.43–33.41 (Gemini 33.41, Claude 25.40, GPT-5.2 33.03). On Level 3 (Tab. 4), the gap widens: code-level remains moderate to good (e.g., LLM-Score 62.04–77.90 across models), but LMM-Score collapses (Gemini 35.97, Claude 16.60, GPT-5.2 , Seed1.5/1.6-VL 18.30/17.43). This demonstrates that while code-level compliance is generally high, it does not guarantee pixel-level visual correctness, making chart-level fidelity the primary bottleneck.

Chart Reproduction Challenge: In Tab. 2, proprietary models still execute but with lower fidelity (e.g., Gemini CFD ER 99.07 with LMM-Score ≈32.85; Claude 93.52/26.44; GPT-5 99.07/35.40). Open-source models suffer larger drops (e.g., InternVL-3-8B 49.07/10.87, Qwen3-VL-30B-A3B 74.07/23.56; several models hit ≈0 ER). Compared to DR/CRD in the same table, CFD exposes weaknesses in Base-Score, LLM-Score and LMM-Score. We concluded reproducing existing charts (CFD) is the hardest subtask in Level1.

Figure 4: Left: Most proprietary and open-source models generalize well on Level 1 and Level 2 tasks when calculating the LLM-score for predicted code assessment. Right: Proprietary models tend to obtain higher LMM-scores on the Level 1 task rather than the Level 2, while open-source models perform poorly on both tasks (scores are lower than 0.5).

Scaling Open-Source Backbones: In Tab. 3, Qwen2.5-VL-72B reaches 74.55 ER with strong code-level, yet chart-level is only 16.69; InternVL-3-38B shows 69.80 ER and similar code-level strength (Base-Score 56.74, LLM-Score 56.98), but chart-level remains 13.32. This contrasts with proprietary models’ ∼90% ER and still-low chart-level (≈17.43–33.41), underscoring that fidelity, not executability, is the persistent gap. These result shows larger open-source backbones close part of the execution gap on level 2, but chart-level fidelity gains are modest.

Thinking Helps Procedural Compliance: On level 2 (Tab. 3), MiMo-VL-7B-RL ER rises from 17.23 → 26.14 when enabling thinking; MiMo-VL-7B-SFT nudges 30.40 → 30.50. LLM-side (code-level GPT-Score) also improves slightly. However, chart-level remains low or mixed (e.g., MiMo-SFT 15.23 → 16.38; MiMo-RL thinking row lacks chart-level). The net effect suggests that chain-of-thought/planning aids procedural compliance, yet post-render pixel-level exactness requires additional mechanisms (e.g., render-then-verify loops). This indicates ”Thinking” variants help instruction following and executability, but visual fidelity improvements are inconsistent.

Metric Analysis: From the results in level1, 2, 3(Tab. 2 3 4), we observe that Base-Score and LLM-Score are highly consistent with each other(e.g.,in Tab. 3, Gemini 70.78/72.21 Qwen2.5-vl-7B 49.22/45.35), yet exhibit a substantial gap compared to LMM-Score. This indicates that both rule-based evaluation and LLM-based code-only assessment correlate poorly with actual visual similarity, revealing a significant mismatch between code-level correctness and pixel-level visual fidelity. To further examine the validity of using LMMs for image similarity evaluation, we additionally conduct human comparison studies; detailed experimental results are reported in Appendix 12.1.

Table-to-Chart Gap: On Level 1 CRD (Tab. 2), multiple models achieve very high ER (e.g., Gemini 100; Cluade 100), yet LMM-Score remains low ( 40.72–40.63 across models). On level 3 (Tab. 4), code-level Visual/Text/Type scores are solid for leading models (e.g., Gemini 84.00/80.45/96.60, GPT-5.2 78.34/76.09/93.81), but chart-level stays around 35.97–16.29, highlighting the gap between semantic correctness and visual exactness. Table to chart is relatively “easy to execute” but still hard to render faithfully.

Failure Mode Analysis: We further analyze model failures across all levels (see Appendix 12.4). Errors are mainly caused by data extraction inaccuracies, visual style mismatches, layout inconsistencies, and axis-related issues. The dominant bottleneck shifts from perception (Level 1) to reasoning (Level 2) and long-context understanding (Level 3), reflecting the increasing complexity of the tasks.

4.3 Discussion.

Model Performance Across Manually Defined Difficulty Levels.

In this experiment, we ask the human labeler to split each level into easy, medium and hard, in total three levels, and each subset contains 30 samples. As shown in Fig. 39, model performance exhibits a clear correlation with manually annotated difficulty levels across all benchmark stages. On Level 1, proprietary models (e.g., GPT-5.2, Gemini-3-Pro, Claude-Sonnet-4) maintain relatively strong scores across Easy, Medium, and Hard subsets, though the overall fidelity remains moderate. In contrast, most open-source models show low scores and struggle particularly on harder cases. On Level 2, performance declines noticeably even for proprietary models, with overall scores dropping to ∼17.43–33.41 and sharper degradation from Easy to Hard, indicating sensitivity to increased editing complexity. By Level 3, almost all models fail regardless of difficulty level: LMM-scores drop to a low range, showing that long-context table-to-chart generation overwhelms current models. These trends suggest that while models can partially track difficulty scaling on simpler tasks, the hardest scenarios effectively collapse their ability to produce faithful visualizations.

Code Generalization Holds, Visual Fidelity Lags.

As shown in Fig. 4, the performance trends differ substantially when measured by LLM-score versus LMM-score. On the left, both proprietary and open-source models generalize reasonably well from Level 1 to Level 2 when evaluated with LLM-score, indicating that code-level syntax and structure can often be preserved across tasks. On the right, however, the LMM-score reveals a sharper divide: proprietary models achieve relatively higher visual fidelity on Level 1 than on Level 2, whereas open-source models perform poorly on both levels, with most scores remaining below 0.5. This contrast highlights that while models can maintain code-level compliance, translating such compliance into pixel-level faithful renderings remains a key unsolved challenge, particularly for open-source systems.

5 Limitations

Overall, Chart2Code has two key limitations. First, all tasks are currently in English, leaving multilingual chart2code as an important direction for future work. Second, our evaluation relies on LLM-based judges to assess code correctness and visual fidelity. While this enables scalable evaluation, it may introduce biases and inaccuracies compared to human assessment, especially for subtle visual details. Future work will explore multilingual expansion and more robust evaluation protocols to improve the benchmark’s coverage and reliability.

6 Conclusion

We presented Chart2Code, a hierarchical benchmark for chart-to-code generation that spans three progressively challenging levels: chart reproduction, chart editing, and long-table–to–chart generation. Through a large-scale evaluation of 29 state-of-the-art LMMs, we observe a clear and consistent trend: although current models perform reasonably well on simple chart reproduction, their performance degrades substantially when faced with complex editing operations and long-context visualization tasks, thereby revealing significant gaps in practical capability. Taken together, these findings highlight the persistent and unsolved challenges in chart-to-code generation, and emphasize the need for future models with stronger reasoning abilities, better generalization, and improved robustness in real-world visualization scenarios.

7 Acknowledgement

This project is supported by the National Natural Science Foundation of China under Grant No 62502544.

References

[1] Anthropic. Introducing claude 4. Preprint, 2025. URL https://www.anthropic.com/news/claude-4.

[2] Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixuan Liu, Dunjie Lu, Ruilin Luo, Chenxu Lv, Rui Men, Lingchen Meng, Xuancheng Ren, Xingzhang Ren, Sibo Song, Yuchong Sun, Jun Tang, Jianhong Tu, Jianqiang Wan, Peng Wang, Pengfei Wang, Qiuyue Wang, Yuxuan Wang, Tianbao Xie, Yiheng Xu, Haiyang Xu, Jin Xu, Zhibo Yang, Mingkun Yang, Jianxin Yang, An Yang, Bowen Yu, Fei Zhang, Hang Zhang, Xi Zhang, Bo Zheng, Humen Zhong, Jingren Zhou, Fan Zhou, Jing Zhou, Yuanzhi Zhu, and Ke Zhu. Qwen3-vl technical report, 2025a. URL https://arxiv.org/abs/2511.21631.

[3] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report, 2025b. URL https://arxiv.org/abs/2502.13923.

[4] Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271, 2024.

[5] Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261, 2025.

[6] Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, et al. Molmo and pixmo: Open weights and open data for state-of-the-art multimodal models. arXiv e-prints, pages arXiv–2409, 2024.

[7] Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Diego Rojas, Guanyu Feng, Hanlin Zhao, Hanyu Lai, Hao Yu, Hongning Wang, Jiadai Sun, Jiajie Zhang, Jiale Cheng, Jiayi Gui, Jie Tang, Jing Zhang, Juanzi Li, Lei Zhao, Lindong Wu, Lucen Zhong, Mingdao Liu, Minlie Huang, Peng Zhang, Qinkai Zheng, Rui Lu, Shuaiqi Duan, Shudan Zhang, Shulin Cao, Shuxun Yang, Weng Lam Tam, Wenyi Zhao, Xiao Liu, Xiao Xia, Xiaohan Zhang, Xiaotao Gu, Xin Lv, Xinghan Liu, Xinyi Liu, Xinyue Yang, Xixuan Song, Xunkai Zhang, Yifan An, Yifan Xu, Yilin Niu, Yuantao Yang, Yueyan Li, Yushi Bai, Yuxiao Dong, Zehan Qi, Zhaoyu Wang, Zhen Yang, Zhengxiao Du, Zhenyu Hou, and Zihan Wang. Chatglm: A family of large language models from glm-130b to glm-4 all tools, 2024.

[8] Google. Gemini3 system card. Preprint, 2025. URL https://deepmind.google/models/gemini/pro/.

[9] Dong Guo, Faming Wu, Feida Zhu, Fuxing Leng, Guang Shi, Haobin Chen, Haoqi Fan, Jian Wang, Jianyu Jiang, Jiawei Wang, et al. Seed1.5-vl technical report. arXiv preprint arXiv:2505.07062, 2025.

[10] Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. SWE-bench: Can language models resolve real-world github issues? In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=VTF8yNQM66.

[11] Jing Yu Koh, Robert Lo, Lawrence Jang, Vikram Duvvur, Ming Chong Lim, Po-Yu Huang, Graham Neubig, Shuyan Zhou, Ruslan Salakhutdinov, and Daniel Fried. Visualwebarena: Evaluating multimodal agents on realistic visual web tasks. arXiv preprint arXiv:2401.13649, 2024.

[12] Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326, 2024a.

[13] Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models. arXiv preprint arXiv:2407.07895, 2024b.

[14] Shawn Li, Ryan Rossi, Sungchul Kim, Sunav Choudhary, Franck Dernoncourt, Puneet Mathur, Zhengzhong Tu, and Yue Zhao. Charts are not images: On the challenges of scientific chart editing, 2025a. URL https://arxiv.org/abs/2512.00752.

[15] Zhen Li, Duan Li, Yukai Guo, Xinyuan Guo, Bowen Li, Lanxi Xiao, Shenyu Qiao, Jiashu Chen, Zijian Wu, Hui Zhang, Xinhuan Shu, and Shixia Liu. Chartgalaxy: A dataset for infographic chart understanding and generation, 2025b. URL https://arxiv.org/abs/2505.18668.

[16] Kevin Qinghong Lin, Linjie Li, Difei Gao, Qinchen WU, Mingyi Yan, Zhengyuan Yang, Lijuan Wang, and Mike Zheng Shou. VideoGUI: A benchmark for GUI automation from instructional videos. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024. URL https://openreview.net/forum?id=jSKtxmxc0M.

[17] Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 26296–26306, June 2024.

[18] Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Hao Yang, Yaofeng Sun, Chengqi Deng, Hanwei Xu, Zhenda Xie, and Chong Ruan. Deepseek-vl: Towards real-world vision-language understanding, 2024.

[19] Ahmed Masry, Do Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. ChartQA: A benchmark for question answering about charts with visual and logical reasoning. In Findings of the Association for Computational Linguistics: ACL 2022, pages 2263–2279, Dublin, Ireland, May 2022. Association for Computational Linguistics. 10.18653/v1/2022.findings-acl.177. URL https://aclanthology.org/2022.findings-acl.177.

[20] Deepak Nathani, Lovish Madaan, Nicholas Roberts, Nikolay Bashlykov, Ajay Menon, Vincent Moens, Mikhail Plekhanov, Amar Budhiraja, Despoina Magka, Vladislav Vorotilov, Gaurav Chaurasia, Dieuwke Hupkes, Ricardo Silveira Cabral, Tatiana Shavrina, Jakob Nicolaus Foerster, Yoram Bachrach, William Yang Wang, and Roberta Raileanu. MLGym: A new framework and benchmark for advancing AI research agents. In Second Conference on Language Modeling, 2025. URL https://openreview.net/forum?id=ryTr83DxRq.

[21] OpenAI. Gpt-5 system card. Preprint, 2025a. URL https://openai.com/index/introducing-gpt-5/.

[22] OpenAI. Gpt-5.2 system card. Preprint, 2025b. URL https://openai.com/index/introducing-gpt-5-2/.

[23] Juan Rodriguez, Xiangru Jian, Siba Smarak Panigrahi, Tianyu Zhang, Aarash Feizi, Abhay Puri, Akshay Kalkunte, François Savard, Ahmed Masry, Shravan Nayak, Rabiul Awal, Mahsa Massoud, Amirhossein Abaskohi, Zichao Li, Suyuchen Wang, Pierre-André Noël, Mats Leon Richter, Saverio Vadacchino, Shubham Agarwal, Sanket Biswas, Sara Shanian, Ying Zhang, Noah Bolger, Kurt MacDonald, Simon Fauvel, Sathwik Tejaswi, Srinivas Sunkara, Joao Monteiro, Krishnamurthy DJ Dvijotham, Torsten Scholak, Nicolas Chapados, Sepideh Kharagani, Sean Hughes, M. Özsu, Siva Reddy, Marco Pedersoli, Yoshua Bengio, Christopher Pal, Issam Laradji, Spandana Gella, Perouz Taslakian, David Vazquez, and Sai Rajeswar. Bigdocs: An open dataset for training multimodal models on document and code tasks, 2025a. URL https://arxiv.org/abs/2412.04626.

[24] Juan A. Rodriguez, Abhay Puri, Shubham Agarwal, Issam H. Laradji, Pau Rodriguez, Sai Rajeswar, David Vazquez, Christopher Pal, and Marco Pedersoli. Starvector: Generating scalable vector graphics code from images and text. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16175–16186, June 2025b.

[25] Kimi Team, Angang Du, Bohong Yin, Bowei Xing, Bowen Qu, Bowen Wang, Cheng Chen, Chenlin Zhang, Chenzhuang Du, Chu Wei, Congcong Wang, Dehao Zhang, Dikang Du, Dongliang Wang, Enming Yuan, Enzhe Lu, Fang Li, Flood Sung, Guangda Wei, Guokun Lai, Han Zhu, Hao Ding, Hao Hu, Hao Yang, Hao Zhang, Haoning Wu, Haotian Yao, Haoyu Lu, Heng Wang, Hongcheng Gao, Huabin Zheng, Jiaming Li, Jianlin Su, Jianzhou Wang, Jiaqi Deng, Jiezhong Qiu, Jin Xie, Jinhong Wang, Jingyuan Liu, Junjie Yan, Kun Ouyang, Liang Chen, Lin Sui, Longhui Yu, Mengfan Dong, Mengnan Dong, Nuo Xu, Pengyu Cheng, Qizheng Gu, Runjie Zhou, Shaowei Liu, Sihan Cao, Tao Yu, Tianhui Song, Tongtong Bai, Wei Song, Weiran He, Weixiao Huang, Weixin Xu, Xiaokun Yuan, Xingcheng Yao, Xingzhe Wu, Xinhao Li, Xinxing Zu, Xinyu Zhou, Xinyuan Wang, Y. Charles, Yan Zhong, Yang Li, Yangyang Hu, Yanru Chen, Yejie Wang, Yibo Liu, Yibo Miao, Yidao Qin, Yimin Chen, Yiping Bao, Yiqin Wang, Yongsheng Kang, Yuanxin Liu, Yuhao Dong, Yulun Du, Yuxin Wu, Yuzhi Wang, Yuzi Yan, Zaida Zhou, Zhaowei Li, Zhejun Jiang, Zheng Zhang, Zhilin Yang, Zhiqi Huang, Zihao Huang, Zijia Zhao, Ziwei Chen, and Zongyu Lin. Kimi-vl technical report, 2025. URL https://arxiv.org/abs/2504.07491.

[26] Seed Team. Seed1.6 tech introduction. Preprint, 2025. URL https://seed.bytedance.com/en/seed1_6.

[27] Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191, 2024a.

[28] Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, et al. Internvl3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508.18265, 2025.

[29] Zirui Wang, Mengzhou Xia, Luxi He, Howard Chen, Yitao Liu, Richard Zhu, Kaiqu Liang, Xindi Wu, Haotian Liu, Sadhika Malladi, Alexis Chevalier, Sanjeev Arora, and Danqi Chen. Charxiv: Charting gaps in realistic chart understanding in multimodal LLMs. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024b. URL https://openreview.net/forum?id=cy8mq7QYae.

[30] Chengyue Wu, Zhixuan Liang, Yixiao Ge, Qiushan Guo, Zeyu Lu, Jiahao Wang, Ying Shan, and Ping Luo. Plot2Code: A comprehensive benchmark for evaluating multi-modal large language models in code generation from scientific plots. In Luis Chiruzzo, Alan Ritter, and Lu Wang, editors, Findings of the Association for Computational Linguistics: NAACL 2025, pages 3006–3028, Albuquerque, New Mexico, April 2025. Association for Computational Linguistics. ISBN 979-8-89176-195-7. 10.18653/v1/2025.findings-naacl.164. URL https://aclanthology.org/2025.findings-naacl.164/.

[31] LCT Xiaomi and Core Team. Mimo-vl technical report. arXiv preprint arXiv:2506.03569, 2025.

[32] Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh Jing Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, Yitao Liu, Yiheng Xu, Shuyan Zhou, Silvio Savarese, Caiming Xiong, Victor Zhong, and Tao Yu. OSWorld: Benchmarking multimodal agents for open-ended tasks in real computer environments. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024. URL https://openreview.net/forum?id=tN61DTr4Ed.

[33] Duo Xu, Hao Cheng, Xin Lin, Zhen Xie, and Hao Wang. Chartm3: A multi-stage code-driven pipeline for constructing multi-dimensional and multi-step visual reasoning data in chart comprehension, 2025. URL https://arxiv.org/abs/2511.02415.

[34] Zhengzhuo Xu, Sinan Du, Yiyan Qi, Chengjin Xu, Chun Yuan, and Jian Guo. Chartbench: A benchmark for complex visual reasoning in charts. ArXiv, abs/2312.15915, 2023. URL https://api.semanticscholar.org/CorpusID:266550948.

[35] Cheng Yang, Chufan Shi, Yaxin Liu, Bo Shui, Junjie Wang, Mohan Jing, Linran XU, Xinyu Zhu, Siheng Li, Yuxiang Zhang, Gongye Liu, Xiaomei Nie, Deng Cai, and Yujiu Yang. Chartmimic: Evaluating LMM’s cross-modal reasoning capability via chart-to-code generation. In The Thirteenth International Conference on Learning Representations, 2025a. URL https://openreview.net/forum?id=sGpCzsfd1K.

[36] John Yang, Carlos E. Jimenez, Alex L. Zhang, Kilian Lieret, Joyce Yang, Xindi Wu, Ori Press, Niklas Muennighoff, Gabriel Synnaeve, Karthik R. Narasimhan, Diyi Yang, Sida I. Wang, and Ofir Press. SWE-bench multimodal: Do ai systems generalize to visual software domains? In The Thirteenth International Conference on Learning Representations, 2025b. URL https://openreview.net/forum?id=riTiq3i21b.

[37] Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik R Narasimhan. {$\tau$}-bench: A benchmark for \underline{T}ool-\underline{A}gent-\underline{U}ser interaction in real-world domains. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=roNSXZpUDN.

[38] Zhehao Zhang, Weicheng Ma, and Soroush Vosoughi. Is GPT-4V (ision) all you need for automating academic data visualization? exploring vision-language models’ capability in reproducing academic charts. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Findings of the Association for Computational Linguistics: EMNLP 2024, pages 8271–8288, Miami, Florida, USA, November 2024. Association for Computational Linguistics. 10.18653/v1/2024.findings-emnlp.485. URL https://aclanthology.org/2024.findings-emnlp.485/.

[39] Henry Hengyuan Zhao, Kaiming Yang, Wendi Yu, Difei Gao, and Mike Zheng Shou. Worldgui: An interactive benchmark for desktop gui automation from any starting point, 2025a. URL https://arxiv.org/abs/2502.08047.

[40] Xuanle Zhao, Xuexin Liu, Haoyue Yang, Xianzhen Luo, Fanhu Zeng, Jianling Li, Qi Shi, and Chi Chen. Chartedit: How far are mllms from automating chart analysis? evaluating mllms’ capability via chart editing. arXiv preprint arXiv:2505.11935, 2025b.

[41] Zhipu. Gpt-4.6 system card. Preprint, 2025. URL https://z.ai/blog/glm-4.6v.

[42] Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479, 2025.

8 LLM Usage Statement

We disclose the use of Large Language Models (LLMs) in this research in several capacities.

First, during the preparation of this manuscript, we utilized an LLM for grammatical correction and stylistic refinement to improve the paper’s readability.

Second, and central to our methodology, multiple LLMs served as the subjects of our experiments to test our proposed benchmark. Furthermore, the evaluation metrics for our benchmark involved using an LLM to assess the comprehensive quality of the results.

We explicitly state that we have never relied on LLMs to generate core research ideas, methodologies, experimental designs, or conclusions. All technical contributions and analyses presented herein are the original work of the authors.

9 User-Centric Case Studies

In this section, we present representative examples that reflect scenarios commonly encountered by real users. Each example includes the input instruction, structured data, reference visualization, and model-generated outputs, allowing for a direct comparison between correct and erroneous results.

The first example corresponds to a Level 2 task ("Error Sample"), where the model is required not only to generate chart code but also to perform data editing and conditional grouping based on the instruction. Specifically, the model must distinguish between sufficient and insufficient inventory categories and reflect this grouping through appropriate visual encoding (e.g., color separation) while preserving the overall chart style. This requires jointly reasoning over data values, instruction constraints, and visual design.

As shown in the figure, most LMMs fail in multiple aspects. Common errors include: (1) incorrect data analysis, where models misinterpret numerical relationships or fail to apply the required grouping logic; (2) visual style inconsistencies, such as failing to reproduce the reference color scheme or incorrectly using a pie chart instead of a donut chart; (3) missing or incomplete components, including absent sub-figures or incorrect legend configurations. These errors indicate that models struggle to maintain consistency between data, instructions, and visual representation.

We observe that even strong models often fail on this seemingly routine setting, which highlights their difficulty in handling tasks that are trivial for humans. In contrast, the correct example demonstrates that solving this task requires precise alignment between data extraction, reasoning, and visual rendering.

Moreover, as illustrated in the subsequent cases ("LLM capability exploration"), existing LMMs frequently produce incorrect outputs even for basic perception tasks, such as identifying chart elements or extracting key information. These failures suggest that current models lack robust visual grounding and fine-grained reasoning abilities.

Overall, these observations indicate that if models cannot reliably handle such user-level scenarios, they are unlikely to succeed in more complex chart2code tasks involving long-context data and compositional instructions.

10 Fine-Grained Visual Perception Analysis

In this section, we analyze the fine-grained visual perception capabilities of LMMs through controlled examples ("LLM capability exploration"), focusing on key attributes such as titles, legends, grid patterns, and axis details.

Each example isolates a specific visual attribute, removing complex reasoning and generation requirements, to directly probe the models’ low-level visual grounding ability. Despite this simplification, even strong models exhibit inconsistent performance. For example, models may confuse title structures (e.g., single-line vs. multi-line), misidentify legend positions, or fail to distinguish visual styles such as dashed versus solid grid lines. They also frequently overlook subtle attributes, including tick label rotation and axis orientation.

These results suggest that current LMMs lack robust fine-grained visual perception, particularly when precise spatial and stylistic details are required. Such limitations are critical, as accurate chart-to-code generation depends on reliably extracting these low-level attributes. Improving visual grounding at this level is therefore essential for advancing chart2code performance.

Instruction: Analyze inventory distribution by category. - Highlight sufficient inventory in ’Grooming Tools’ and ’Kids’ Clothing’ - Highlight insufficient inventory in ’Toys & Games’ and ’Books & Stationery’ - Use separate colored sections in the chart to distinguish these two groups Generate runnable Python code matching the uploaded image style.

Data text:

Reference Figure [Uncaptioned image]

GT Figure [Uncaptioned image]

Question: Please, based on this image, tell me if there is any title text in the picture? If so, what is its position, is it single or multiple lines, and what are its size and color?

Figure:

Question: Please describe the arrangement and relative position of the legend in this chart: Is it arranged horizontally, vertically, or in a grid? Which side of the main graph is it located on (top, bottom, left, right, or embedded within)?

Figure:

Question: Please describe the grid lines in this chart: Are they horizontal, vertical, or both? Are the lines dashed or solid?

Figure:

Question: Please describe the primary tick marks on the axes of this chart: whether they exist, their thickness and orientation (facing outward or inward), as well as the position and rotation angle of the tick labels.

Figure:

11 Data Curation

To construct a comprehensive and challenging chart benchmark, we collected a rich dataset of chart images and their corresponding raw data from multiple sources.

11.1 Data Curation and Annotation

11.1.1 Data curation

Chart Data: Our chart figure sources primarily consist of three aspects. First, we collected approximately 5,000 paper charts from Arxiv, spanning from January 2024 to July 2025, covering various fields such as CSEE, Physics, Statistics, and Economics, to ensure diversity and modernity in the chart types. Second, we gathered 1,000 example charts from function libraries such as Matplotlib, Seaborn, WordCloudX, Scipy, as well as Matlab plotting example tutorials. Finally, we filtered 150 difficult charts from the ChartMimic [35] dataset.

Raw Data: Our benchmark collects raw data from sources such as Kaggle, Annual Reports, and publicly available data from various company websites. The raw data includes Excel spreadsheets, figures, text, and other formats, covering multiple domains such as corporate financial reports, flight route data, weather data, GDP data, and car sales figures. Additionally, we have intentionally selected data of varying lengths to test the LLM’s ability to analyze and process long text data.

11.1.2 Data filtering

Chart Data: We propose a "gather-distribution" data selection process. First, we gather data from various sources into a chart pool, which is then roughly filtered by 10 undergraduate computer science students based on chart type and information complexity. Based on this initial selection, we reduce the data to 3,000 charts to ensure that the resulting dataset contain a diverse range of visual elements and chart types. Next, the gathered data is distributed by category to 5 experts with many years of experience in Python plotting for independent evaluation. The evaluation criteria are refined into three dimensions: data complexity, visual complexity, and programming complexity. Each dimension is independently assessed to select more valuable charts as part of the benchmark data. Finally, the charts from various categories are aggregated to form the 719 unique reference figures in the benchmark.

Raw Data: Since the raw data we collected contains various data formats, we first use automated scripts to filter out the raw data that exhibits rich numerical performance and is suitable for plotting. After that, we conduct manual checks to preserve the diversity of the raw data as much as possible. The final selection includes 71 Excel files, 108 raw data figures, and 36 raw data text files.

11.1.3 Data Annotation

During the data annotation process for the three-level tasks, we employed an interactive data annotation method based on Python scripts and agents, which we refer to as the human-AI interactive annotation process. Specifically, in the level 1 data annotation process, annotators, with the assistance of the LMM, recreate the selected data by writing Python code. The data generated here directly serves as the first setting of the Level 1 task. Subsequently, based on the 719 scripts, annotators select and modify suitable chart types using the data from the 108 raw table figures and 36 raw table text files, resulting in 108+36 customized entries for the second setting of the task.

In the Level 2 annotation process, annotators first categorize and summarize chart editing operations commonly encountered in real-world scenarios. They then modify the code with the help of prompt engineering and Python code injection, leveraging the programming capabilities of LLM. While the LLM may lack proficiency in the chart2code task, its programming ability is exceptional. Through this process, we obtained over 4,700 edited and modified scripts, which were further filtered through the data selection process, ultimately yielding 1,010 high-quality Level 2 data entries.

For Level 3 data annotation, annotators first analyzed the contents of 71 diverse data tables to formulate specific statistical requirements. Leveraging LLMs to generate corresponding preprocessing scripts, they performed systematic data cleaning and transformation. Subsequently, target data aligned with the predefined statistical needs were extracted to facilitate the LLM in generating final plotting code. This rigorous pipeline ultimately yielded 313 high-quality Level 3 data entries.

11.2 Chart Image Data

Data Scource: Our chart image library is primarily composed of three parts, designed to cover a wide range of chart types, visual styles, and information densities.

Charts from Academic Literature: We extracted chart images from approximately 5,000 PDF documents by crawling and parsing papers from the preprint server arXiv using automated scripts. These publications span from January 2024 to July 2025 and cover multiple disciplines, including computer science, physics, statistics, and economics, timestamps distribution of chart sources from arxiv preprint Fig.5. This ensures that our dataset not only includes common statistical charts but also covers the highly customized and information-dense visualizations frequently found in academic research, guaranteeing both diversity and state-of-the-art relevance. Due to their superior data quality and structural complexity, academic charts were prioritized for our dataset construction. Following a final filtration stage, 548 charts were curated from arXiv papers. These instances represent the core of our benchmark, providing the necessary depth for evaluating sophisticated chart-to-code generation.

Examples from Programming Communities and Tutorials: To enrich the diversity of our dataset, we curated 1,000 example charts from the official documentation and tutorials of mainstream visualization libraries, including Matplotlib, Seaborn, Plotly, WordCloudX, Scipy, and MATLAB, as well as various active developer forums. By applying systematic data processing and augmentation techniques, we further expanded the variety and complexity of these instances. This integration significantly broadens the taxonomic coverage of our benchmark, ensuring a more comprehensive evaluation of chart-to-code generation across diverse styles and formats. After final data screening, 87 instances were selected from various programming communities and tutorials. The inclusion of these charts substantially enriches our task categories, ensuring that the benchmark covers a more extensive range of practical visualization patterns.

Existing Chart Datasets: Additionally, 150 high-quality instances were sourced from ChartMimic[35] for inclusion in our initial pool. After a final round of filtering, 50 samples were retained for the final benchmark dataset

Figure 5: Timestamps distribution of chart sources from arxiv preprint.

Preliminary Collection and Deduplication: First, we gathered all charts from the three aforementioned sources into a unified database. We then performed preliminary automated deduplication and format standardization.

Coarse Filtering: We recruited 10 senior undergraduate students majoring in computer science to conduct an initial screening of the chart pool. The screening criteria were primarily based on the clarity of the chart type (i.e., whether it is a common chart type) and its information complexity (e.g., the number of data series, density of text labels). This stage aimed to quickly eliminate ambiguous, overly simplistic, or low-quality images, reducing the dataset size from approximately 6,300 to 3,000.

Expert Evaluation and Annotation: We invited five doctoral students and researchers, each with over three years of experience in data visualization, to serve as experts for a fine-grained evaluation of the filtered charts. We assigned the charts to the experts by category (e.g., line charts, bar charts, scatter plots) and asked them to independently score each chart from 1 to 5 across three dimensions: Data Complexity: Refers to the dimensional and structural complexity of the underlying data required for the chart. Visual Complexity: Refers to the richness of visual elements in the chart, such as markers, colors, annotations, and dual axes. Programming Complexity: Refers to the programming skills and volume of code required to reproduce the chart, such as the need for complex layouts or custom functions. Final Adjudication: We selected charts that achieved a high composite score across the three dimensions and had high inter-rater agreement (>0.8). For charts with disagreements, two core researchers made the final decision. Through this process, we finalized a set of 719 high-quality reference charts.

Table 5: Statistics of Annotation Tasks per Annotator

Annotator Raw Data Chart Data

Annotator 1 20 diverse data tables and Excels. 50 charts.

Annotator 2 20 diverse data tables and Excels. 50 charts.

Annotator 3 50 diverse data figures and Excels. 50 charts.

Annotator 4 50 diverse data figures and Excels. 50 charts.

Annotator 5 50 diverse data tables and Excels. 150 charts.

Annotator 6 50 diverse data tables and Excels. 150 charts.

Annotator 7 40 diverse data tables and Excels. 150 charts.

Annotator 8 40 diverse data tables and Excels. 150 charts.

Annotator 9 check data. check charts.

Annotator 10 check data. check charts.

Annotator 11 check data. check charts.

Annotator 12 check data. check charts.

11.3 Raw Data Filtering

Automated Preprocessing: We developed automated scripts to parse raw data files in various formats (e.g., Excel, CSV, TXT, JSON). These scripts prioritized the selection of data tables that contain abundant numerical, time-series, or categorical information suitable for visualization.

Manual Verification and Diversity Preservation: Subsequently, we manually reviewed the data filtered by the scripts, discarding any incomplete or poorly formatted data. During this process, we placed special emphasis on preserving the diversity of data sources and domains to ensure the final dataset was not biased towards any specific field. Ultimately, we constructed a raw database containing 71 Excel files, 108 structured data figures, and 36 semi-structured text files.

11.4 Human Annotation Details and Ethical Considerations

Annotation Process and Instructions. To ensure high-quality data labeling, we employed a human-in-the-loop approach. Due to the dynamic and iterative nature of the task, static instruction documents were not the primary mode of guidance. Instead, annotators were guided through interactive training sessions and real-time feedback loops to align with the specific research goals. Consequently, the full text of static instructions or screenshots is not included in this report. The annotation criteria focused on high-quality chart and human natural language instruction, and ambiguity was resolved through consensus discussions among the annotators and the authors.

Recruitment and Compensation. The annotation task was performed by undergraduate student. The participants were recruited based on their expertise in coding. They were compensated at a rate that exceeds the local minimum wage and is commensurate with standard industry rates for similar tasks. Some annotators were the authors of this paper, and thus no monetary compensation was involved.

Consent and Ethics. All participants were fully informed of the purpose of the study and the usage of the annotated data. Since the task involved standard data processing with no intervention in the participants’ personal lives and utilized non-sensitive public data, formal approval from an Institutional Review Board (IRB) was not sought. The study adheres to the ethical guidelines regarding human subject participation, ensuring anonymity and the right to withdraw from the task at any time.

12 More Analysis

12.1 Details Evaluation results

The detail Base-Score and LLM-Score results on DR, CRD and CFD as show in Tab. 6, Tab. 7, Tab. 8, Tab. 9 and Tab. 10, Tab. 11 Fig. 40, Fig. 41, Fig. 42, Fig. 43, and Fig. 44 present the comparative results of human scores versus model scores.

Table 6: Details Evaluation results on level-1 Direct mimic result(Base-Score Details)

Model Exec. Rate Code-Level Chart-Level

Color Grid Layout Legend Visual Data Text Type Base LLM LMM

Proprietary

Gemini-3-Pro 97.50 61.92 83.49 84.14 87.17 83.78 68.47 93.59 93.60 78.65 75.86 45.42

Claude-Sonnet-4 96.52 43.69 51.02 77.15 80.39 74.97 55.45 85.66 88.75 65.60 56.65 32.36

GPT-5.2 97.08 65.77 84.94 83.96 87.93 85.21 69.25 93.35 93.85 79.91 77.88 43.73

Seed1.5-VL 87.34 37.85 65.79 76.16 75.22 71.77 53.27 81.05 86.40 63.85 53.84 26.40

Seed1.6-VL 82.61 36.71 61.25 76.87 77.18 69.39 50.62 84.69 80.49 62.46 50.85 26.16

Open-Source LMMs (non-thinking)

LLaVA-OV-Qwen2-7B-SI 82.48 7.84 46.00 58.76 29.25 41.43 17.06 44.72 43.36 31.33 16.81 2.58

LLaVA-OV-Qwen2-7B-OV 72.60 7.61 48.06 65.10 36.51 45.29 19.41 50.00 44.27 34.33 54.85 3.42

DeepSeek-VL-7B 36.44 10.87 49.77 64.69 35.77 49.06 22.62 47.18 61.49 37.49 22.02 3.37

kimi-VL-A3B 70.79 26.93 58.47 69.63 61.68 63.57 38.50 64.72 78.73 52.76 40.36 14.15

Qwen2-VL-7B 62.45 13.16 42.16 69.30 46.14 53.60 25.63 56.09 65.04 40.99 28.74 7.02

Qwen2-VL-72B 74.97 23.44 57.84 68.96 61.70 60.72 35.52 64.78 75.42 50.74 38.27 12.75

InternVL-2.5-8B 67.73 16.46 49.69 68.14 46.18 54.48 28.58 57.76 61.26 42.76 27.49 7.80

InternVL-2.5-38B 85.67 24.22 56.99 71.47 61.88 62.49 38.12 65.69 76.47 51.97 39.83 15.18

InternVL-3-8B 29.49 17.69 48.13 74.59 53.51 58.84 33.35 59.50 69.13 46.58 55.68 9.20

InternVL-3-38B 85.26 27.20 54.62 71.38 60.99 64.89 41.15 68.68 78.44 53.57 42.85 16.68

GLM-4V-9B 54.24 6.01 32.64 63.56 39.80 44.56 18.11 48.69 47.50 32.50 23.12 4.80

GLM-4.6v-flash 54.52 40.68 70.91 78.12 77.96 75.47 55.51 80.94 90.95 66.67 55.38 31.70

Intern-VL-3.5-8B 72.18 26.02 55.78 69.38 56.38 65.72 39.38 64.86 77.15 52.01 41.29 15.41

InternVL-3.5-38B 84.01 31.37 56.41 71.04 62.63 66.82 43.84 70.42 80.06 55.78 42.54 19.78

MiMo-VL-7B-RL 43.12 40.56 61.70 76.42 76.89 73.56 52.02 81.04 89.30 64.41 49.82 21.43

MiMo-VL-7B-SFT 51.74 39.96 59.75 75.99 75.20 73.44 53.71 80.75 87.66 63.99 49.07 22.61

Qwen2.5-VL-7B 69.26 21.55 49.85 71.02 53.45 59.62 35.68 61.97 71.63 48.20 35.46 10.54

Qwen2.5-VL-72B 63.84 35.69 56.65 74.20 68.78 69.69 46.60 74.20 82.36 59.05 47.95 20.97

Molmo-7B-D 4.31 10.77 33.49 45.16 32.72 31.72 8.33 34.02 37.31 25.26 11.48 2.42

Qwen3-VL-30B-A3B 72.18 39.22 59.22 76.61 73.06 73.32 51.40 78.99 88.61 63.09 50.84 23.83

Qwen3-32B 76.91 39.61 55.76 76.55 78.05 71.72 50.82 83.03 85.46 63.10 54.87 18.45

Open-Source LMMs (thinking)

MiMo-VL-7B-RL 21.14 42.06 74.48 88.16 82.21 78.56 59.78 83.82 93.57 70.45 60.00 30.04

MiMo-VL-7B-SFT 41.03 37.29 69.56 83.56 79.00 73.29 53.67 80.76 86.79 65.49 54.33 16.95

Qwen3-VL-30B-A3B 76.08 38.53 58.93 75.88 73.04 72.41 50.00 78.75 87.95 62.37 55.48 15.03

Table 7: Details Evaluation results on level-1 Direct mimic result(LLM-Score Details)

Model Exec. Rate Code-Level Chart-Level

Color Grid Layout Legend Visual Data Text Type Base LLM LMM

Proprietary

Gemini-3-Pro 97.50 75.54 87.78 77.15 88.02 67.22 58.05 84.86 86.36 78.65 75.86 45.42

Claude-Sonnet-4 96.52 53.41 69.56 61.63 76.64 50.57 34.23 70.69 62.12 65.60 56.65 32.36

GPT-5.2 97.08 76.95 88.58 80.08 91.06 70.60 59.80 85.75 89.20 79.91 77.88 43.73

Seed1.5-VL 87.34 48.15 71.25 56.46 72.23 44.90 32.40 67.67 64.78 63.85 53.84 26.40

Seed1.6-VL 82.61 44.69 67.97 55.13 74.31 42.34 28.93 67.09 54.45 62.46 50.85 26.16

Open-Source LMMs (non-thinking)

LLaVA-OV-Qwen2-7B-SI 82.48 15.92 35.59 14.67 25.83 11.79 2.50 34.87 8.52 31.33 16.81 2.58

LLaVA-OV-Qwen2-7B-OV 72.60 50.68 67.82 58.52 76.04 49.30 31.10 69.34 63.94 34.33 54.85 3.42

DeepSeek-VL-7B 36.44 24.41 46.58 20.73 36.32 16.95 4.18 34.22 8.19 37.49 22.02 3.37

kimi-VL-A3B 70.79 41.94 60.95 42.52 58.49 31.85 18.16 54.24 35.37 52.76 40.36 14.15

Qwen2-VL-7B 62.45 28.82 50.61 27.27 44.90 22.22 9.91 47.36 17.59 40.99 28.74 7.02

Qwen2-VL-72B 74.97 34.97 60.72 36.60 59.18 29.96 17.24 56.57 35.20 50.74 38.27 12.75

InternVL-2.5-8B 67.73 27.61 49.61 24.36 44.78 20.55 9.55 46.05 15.25 42.76 27.49 7.80

InternVL-2.5-38B 85.67 37.61 60.33 39.60 60.02 32.22 18.09 55.46 39.27 51.97 39.83 15.18

InternVL-3-8B 29.49 51.83 68.48 60.00 76.09 49.42 31.67 49.42 65.32 46.58 55.68 9.20

InternVL-3-38B 85.26 39.86 61.51 42.66 61.75 34.26 22.32 58.01 45.91 53.57 42.85 16.68

InternVL-3.5-38B 84.01 38.52 60.64 41.42 59.30 34.12 23.61 61.58 44.30 55.78 42.54 19.78

GLM-4V-9B 54.24 26.33 39.44 18.47 38.35 16.92 7.63 39.97 10.13 32.50 23.12 4.80

GLM-4.6v-Flash 54.52 52.26 75.31 58.02 75.24 46.41 32.26 67.76 62.08 66.67 55.38 31.70

Intern-VL-3.5-8B 72.18 40.52 61.23 40.89 56.97 33.34 22.37 56.55 38.16 52.01 41.29 15.41

MiMo-VL-7B-RL 43.12 45.85 68.55 52.92 71.35 41.71 26.15 67.41 52.27 64.41 49.82 21.43

MiMo-VL-7B-SFT 51.74 46.01 69.97 51.94 68.70 41.22 26.19 64.60 49.87 63.99 49.07 22.61

Qwen2.5-VL-7B 69.26 32.16 55.19 34.80 55.03 28.31 16.46 53.78 30.24 48.20 35.46 10.54

Qwen2.5-VL-72B 63.84 46.10 64.36 47.97 68.66 40.05 26.83 62.88 49.67 59.05 47.95 20.97

Molmo-7B-D 4.31 11.61 22.90 10.48 26.45 6.45 1.13 22.10 0.97 25.26 11.48 2.42

Qwen3-VL-30B-A3B 72.18 47.46 68.09 54.27 70.81 44.26 28.21 65.92 54.64 63.09 50.84 23.83

Qwen3-VL-32B 76.91 50.73 67.54 58.69 76.17 48.46 31.06 69.51 64.79 63.10 54.87 18.45

Open-Source LMMs (thinking)

MiMo-VL-7B-RL 21.14 54.41 78.59 64.11 77.14 47.66 40.41 72.24 70.59 70.45 60.00 30.04

MiMo-VL-7B-SFT 41.03 49.06 74.81 57.98 74.57 44.13 31.72 68.33 63.79 65.49 54.33 16.95

Qwen3-VL-30B-A3B 76.08 51.87 68.34 59.39 76.65 49.08 32.24 69.14 63.97 62.37 55.48 15.03

Table 8: Details Evaluation results on level-1 Customize Raw Data result(Base-Score Details)

Model Exec. Rate Code-Level Chart-Level

Color Grid Layout Legend Visual Data Text Type Base LLM LMM

Proprietary

Gemini-3-Pro 100.00 47.55 81.25 80.56 54.34 78.54 66.00 76.10 94.44 69.23 69.76 40.72

Claude-Sonnet-4 100.00 39.06 50.52 85.19 32.70 74.51 65.22 67.70 95.37 61.46 55.03 40.63

GPT-5.2 97.22 45.29 78.93 82.86 38.79 78.83 64.71 72.25 91.43 66.31 65.51 39.26

Seed1.5-VL 97.22 34.89 67.86 87.14 45.53 74.32 72.14 71.55 97.14 65.76 58.73 34.74

Seed1.6-VL 94.44 30.76 59.44 83.82 33.47 74.55 63.89 72.24 94.12 60.69 57.87 28.44

Open-Source LMMs (non-thinking)

LLaVA-OV-Qwen2-7B-SI 19.44 30.49 23.21 85.71 0.00 71.74 49.76 56.12 100.00 49.73 35.93 15.14

LLaVA-OV-Qwen2-7B-OV 8.33 22.92 0.00 66.67 13.98 45.32 23.22 41.94 66.67 32.68 15.67 6.00

DeepSeek-VL-7B 55.56 16.67 67.50 77.50 28.58 64.17 52.16 45.18 86.67 50.73 29.27 9.25

kimi-VL-A3B 66.67 22.59 36.98 81.25 29.28 67.21 46.50 55.56 94.44 50.29 46.85 33.04

Qwen2-VL-7B 77.78 20.25 63.39 85.71 36.01 64.19 48.49 53.13 82.14 52.21 44.50 19.61

Qwen2-VL-72B 86.11 22.22 62.90 90.32 37.34 68.12 55.42 53.79 92.47 56.02 46.89 28.23

InternVL-2.5-8B 69.44 16.61 69.00 96.00 26.42 69.19 52.77 52.92 89.33 54.16 42.80 25.88

InternVL-2.5-38B 86.11 22.85 65.32 88.71 29.53 70.70 55.40 58.51 88.17 55.74 47.98 24.71

InternVL-3-8B 36.11 12.86 73.08 76.92 37.02 62.84 51.94 65.45 89.74 53.47 31.38 6.23

InternVL-3-38B 86.11 23.00 67.74 82.26 35.38 69.99 60.67 61.15 97.85 58.17 51.10 34.00

GLM-4V-9B 75.00 15.14 58.33 83.95 27.78 66.36 48.62 54.36 85.19 50.35 42.91 19.70

Glm-4.6v-flash 61.11 33.54 84.66 81.82 31.68 73.35 71.08 61.22 92.42 63.44 52.75 32.45

Intern-VL-3.5-8B 69.44 4.05 69.73 65.33 11.50 52.01 28.69 25.33 89.33 37.87 14.58 1.68

InternVL-3.5-38B 86.11 23.66 70.97 88.71 44.07 70.77 63.20 64.79 93.55 60.66 53.16 38.83

MiMo-VL-7B-RL 66.67 34.85 51.14 75.00 46.20 73.42 67.99 57.95 87.88 59.73 52.21 25.38

MiMo-VL-7B-SFT 75.00 25.82 37.50 71.88 47.54 70.34 55.06 61.05 83.33 53.34 37.80 22.74

Qwen2.5-VL-7B 83.33 22.51 69.58 88.33 35.67 68.37 52.26 60.51 82.22 55.42 46.72 20.14

Qwen2.5-VL-72B 94.44 26.14 67.89 80.88 34.56 68.85 61.67 59.25 94.12 58.12 50.97 27.53

Molmo-7B-D 2.78 50.07 100.00 100.00 0.00 71.24 78.29 75.00 100.00 70.30 40.00 4.00

Qwen3-VL-30B-A3B 69.44 32.93 67.50 76.67 32.26 65.35 54.57 56.23 85.33 55.84 52.26 23.28

Qwen3-VL-32B 77.78 34.69 70.09 80.36 36.22 72.69 60.29 65.07 90.48 60.49 52.64 36.79

Open-Source LMMs (thinking)

MiMo-VL-7B-RL 75.00 37.93 63.43 88.89 48.64 72.28 67.62 66.43 95.06 64.58 57.96 24.96

MiMo-VL-7B-SFT 77.78 27.23 70.98 85.71 39.14 72.44 66.65 63.25 94.05 61.33 53.89 29.93

Qwen3-VL-30B-A3B 61.11 33.61 71.59 88.64 39.56 74.41 69.98 69.46 96.97 64.78 57.63 35.00

Table 9: Details Evaluation results on level-1 Customize Raw Data result(LLM-Score Details)

Model Exec. Rate Code-Level Chart-Level

Color Grid Layout Legend Visual Data Text Type Base LLM LMM

Proprietary

Gemini-3-Pro 100.00 65.56 85.06 64.58 72.50 61.81 76.06 56.67 73.75 69.23 69.76 40.72

Claude-Sonnet-4 100.00 44.03 64.58 52.08 54.86 48.19 64.44 46.67 66.94 61.46 55.03 40.63

GPT-5.2 97.22 60.71 82.14 62.29 68.00 61.43 65.71 62.00 66.43 66.31 65.51 39.26

Seed1.5-VL 97.22 37.00 72.14 55.43 61.43 45.29 76.00 52.57 74.43 65.76 58.73 34.74

Seed1.6-VL 94.44 39.41 72.94 58.97 59.26 49.85 67.65 55.44 68.09 60.69 57.87 28.44

Open-Source LMMs (non-thinking)

LLaVA-OV-Qwen2-7B-SI 19.44 30.00 40.00 35.00 30.00 34.29 40.00 43.57 36.43 49.73 35.93 15.14

LLaVA-OV-Qwen2-7B-OV 8.33 13.33 33.33 13.33 26.67 16.67 6.67 26.67 0.00 32.68 15.67 6.00

DeepSeek-VL-7B 55.56 21.00 67.50 29.75 31.75 26.50 23.50 19.25 29.00 50.73 29.27 9.25

kimi-VL-A3B 66.67 37.92 61.46 45.83 51.88 38.12 52.71 43.12 46.88 50.29 46.85 33.04

Qwen2-VL-7B 77.78 31.25 74.64 39.11 48.04 32.32 54.11 41.61 38.57 52.21 44.50 19.61

Qwen2-VL-72B 86.11 33.87 64.52 43.87 44.84 37.90 60.48 42.10 46.94 56.02 46.89 28.23

InternVL-2.5-8B 69.44 29.20 69.80 40.80 40.40 31.80 54.40 38.00 40.00 54.16 42.80 25.88

InternVL-2.5-38B 86.11 29.03 73.23 44.03 46.61 37.90 61.45 46.45 50.65 55.74 47.98 24.71

InternVL-3-8B 36.11 20.00 73.85 29.62 38.46 26.54 25.00 26.15 29.23 53.47 31.38 6.23

InternVL-3-38B 86.11 34.68 72.90 47.58 51.45 40.97 67.10 45.00 49.52 58.17 51.10 34.00

InternVL-3.5-38B 86.11 34.35 71.13 54.35 51.45 43.23 67.74 46.61 60.65 60.66 53.16 38.83

GLM-4V-9B 75.00 32.78 70.74 35.37 44.63 32.78 56.11 38.33 29.44 50.35 42.91 19.70

GLM-4.6v-Flash 61.11 37.73 87.05 52.27 51.82 41.14 56.14 44.55 62.95 63.44 52.75 32.45

Intern-VL-3.5-8B 69.44 13.00 51.80 14.00 14.20 14.80 0.00 3.60 21.40 37.87 14.58 1.68

MiMo-VL-7B-RL 66.67 43.12 69.17 45.62 59.17 46.67 59.79 42.29 53.33 59.73 52.21 25.38

MiMo-VL-7B-SFT 75.00 28.33 61.30 40.93 46.30 34.81 37.41 32.22 30.93 53.34 37.80 22.74

Qwen2.5-VL-7B 83.33 29.33 74.00 43.33 50.33 36.33 57.00 44.50 46.00 55.42 46.72 20.14

Qwen2.5-VL-72B 94.44 36.18 73.09 44.12 52.35 42.21 63.97 45.29 52.35 58.12 50.97 27.53

Molmo-7B-D 2.78 25.00 100.00 40.00 0.00 10.00 30.00 40.00 100.00 70.30 40.00 4.00

Qwen3-VL-30B-A3B 69.44 39.40 75.00 51.20 51.60 43.60 61.60 40.20 59.00 55.84 52.26 23.28

Qwen3-Dense-32B 77.78 40.89 80.00 51.43 54.64 43.75 58.93 48.21 48.75 60.49 52.64 36.79

Open-Source LMMs (thinking)

MiMo-VL-7B-RL 75.00 43.15 73.89 56.30 62.78 45.56 72.78 47.59 61.67 64.58 57.96 24.96

MiMo-VL-7B-SFT 77.78 35.36 77.50 51.79 51.79 42.32 65.54 47.32 66.43 61.33 53.89 29.93

Qwen3-VL-30B-A3B 61.11 47.27 79.23 55.68 60.00 47.05 64.32 50.91 60.23 64.78 57.63 35.00

Table 10: Details Evaluation results on level-1 Customize Figure Data result(Base-Score Details)

Model Exec. Rate Code-Level Chart-Level

Color Grid Layout Legend Visual Data Text Type Base LLM LMM

Proprietary

Gemini-3-Pro 99.07 52.32 75.31 86.45 63.33 81.58 62.75 77.16 93.86 70.78 71.12 32.85

Claude-Sonnet-4 93.52 48.84 52.64 85.48 58.65 79.04 57.41 71.10 93.30 65.27 65.99 26.44

GPT-5.2 99.07 61.93 78.66 85.51 61.32 83.12 62.66 75.46 96.95 73.02 71.42 35.40

Seed1.5-VL 79.63 48.02 59.30 87.21 57.02 74.28 57.13 75.32 92.36 65.58 64.19 19.53

Seed1.6-VL 79.63 45.99 64.97 83.14 64.99 76.02 57.72 74.38 91.28 66.22 63.12 25.51

Open-Source LMMs (non-thinking)

LLaVA-OV-Qwen2-7B-SI 0.00 - - - - - - - - - - 0.00

LLaVA-OV-Qwen2-7B-OV 0.00 - - - - - - - - - - 0.00

DeepSeek-VL-7B 4.63 1.15 40.00 80.00 20.00 52.25 17.00 35.15 40.00 30.37 25.20 5.60

kimi-VL-A3B 64.81 35.14 47.80 80.71 40.63 66.55 43.79 58.94 83.76 53.62 45.36 16.41

Qwen2-VL-7B 31.48 20.76 53.31 79.41 45.90 68.20 41.07 58.34 81.37 51.02 41.03 8.65

Qwen2-VL-72B 64.81 38.86 52.38 87.86 58.18 71.23 47.31 65.35 89.24 59.66 56.41 15.81

InternVL-2.5-8B 37.04 25.92 54.69 86.25 37.60 66.06 39.15 53.18 87.92 51.58 38.81 12.85

InternVL-2.5-38B 77.78 37.53 61.90 84.13 50.74 73.02 50.31 63.71 89.44 59.86 55.51 22.88

InternVL-3-8B 12.96 23.97 43.75 82.14 37.88 69.88 38.55 48.62 100.00 50.73 35.61 3.14

InternVL-3-38B 36.11 37.10 66.99 87.18 51.74 69.71 49.32 68.27 84.96 60.17 60.42 18.56

InternVL-3.5-38B 24.07 38.85 61.70 90.38 58.57 74.83 46.18 68.27 87.69 61.15 60.69 21.85

GLM-4V-9B 41.67 5.57 40.83 75.56 17.78 47.17 21.21 28.22 53.33 31.64 12.84 -

GLM-4.6v-Flash 55.56 47.47 61.94 90.00 61.25 78.09 52.89 70.49 93.89 65.64 60.23 22.57

Intern-VL-3.5-8B 49.07 29.86 53.85 83.02 43.76 63.91 40.18 60.74 86.42 53.18 48.85 10.87

MiMo-VL-7B-RL 42.59 51.85 62.23 79.35 55.70 78.59 53.98 70.77 85.58 64.39 57.68 21.44

MiMo-VL-7B-SFT 32.41 44.99 58.47 75.81 59.95 81.79 57.63 68.12 97.42 64.68 54.74 22.83

Qwen2.5-VL-7B 44.44 27.43 57.12 73.61 38.66 61.61 38.58 53.06 86.94 50.30 41.78 7.81

Qwen2.5-VL-72B 47.22 45.18 59.56 84.31 56.47 77.20 55.32 68.95 91.96 63.95 61.37 24.48

Molmo-7B-D 1.85 5.68 62.50 100.00 0.00 65.44 40.01 36.49 100.00 45.58 24.00 0.00

Qwen3-VL-30B-A3B 74.07 47.15 53.18 85.42 53.22 74.28 52.70 72.34 92.04 63.02 60.23 26.34

Qwen3-Dense-32B 80.56 51.43 64.80 86.21 60.36 75.79 56.32 73.16 90.65 66.64 65.27 25.84

Open-Source LMMs (thinking)

MiMo-VL-7B-RL 37.96 57.15 58.23 90.65 53.93 77.89 59.95 73.84 91.79 68.05 64.38 29.82

MiMo-VL-7B-SFT 37.96 48.44 56.61 91.06 60.84 74.02 49.44 71.76 90.41 64.04 56.04 23.29

Qwen3-VL-30B-A3B 80.56 49.78 62.02 87.36 53.82 78.44 58.00 72.61 95.94 66.57 62.86 23.56

Table 11: Details Evaluation results on level-1 Customize Figure Data result(LLM-Score Details)

Model Exec. Rate Code-Level Chart-Level

Color Grid Layout Legend Visual Data Text Type Base LLM LMM

Proprietary

Gemini-3-Pro 99.07 68.90 87.67 73.10 79.67 66.95 65.81 62.38 85.55 70.78 71.12 32.85

Claude-Sonnet-4 93.52 61.55 76.18 70.60 71.90 60.85 58.95 60.25 85.70 65.27 65.99 26.44

GPT-5.2 99.07 68.91 89.04 71.47 79.98 67.74 64.29 62.38 83.96 73.02 71.42 35.40

Seed1.5-VL 79.63 58.59 78.27 65.15 72.35 55.35 61.29 60.47 78.06 65.58 64.19 19.53

Seed1.6-VL 79.63 56.37 78.75 67.18 72.62 56.73 63.93 58.69 71.67 66.22 63.12 25.51

Open-Source LMMs (non-thinking)

LLaVA-OV-Qwen2-7B-SI 0.00 - - - - - - - - - - 0.00

LLaVA-OV-Qwen2-7B-OV 0.00 - - - - - - - - - - 0.00

DeepSeek-VL-7B 4.63 28.00 56.00 32.00 42.00 19.00 0.00 21.00 26.00 30.37 25.20 5.60

kimi-VL-A3B 64.81 45.64 68.50 48.00 55.00 40.86 32.50 47.00 37.93 53.62 45.36 16.41

Qwen2-VL-7B 31.48 45.74 71.03 39.41 51.62 35.88 26.03 37.50 31.32 51.02 41.03 8.65

Qwen2-VL-72B 64.81 52.14 75.29 57.21 67.00 48.57 48.79 53.07 61.14 59.66 56.41 15.81

InternVL-2.5-8B 37.04 38.97 66.28 42.82 41.92 36.28 26.92 43.33 35.64 51.58 38.81 12.85

InternVL-2.5-38B 77.78 51.49 75.65 59.23 65.65 47.98 45.48 51.07 61.55 59.86 55.51 22.88

InternVL-3-8B 12.96 40.00 62.86 41.43 45.00 37.86 9.64 35.71 33.93 50.73 35.61 3.14

InternVL-3-38B 36.11 59.10 79.74 61.28 66.28 52.69 54.62 55.00 61.79 60.17 60.42 18.56

InternVL-3.5-38B 24.07 55.58 77.31 54.23 68.27 50.00 65.00 52.69 63.27 61.15 60.69 21.85

GLM-4V-9B 41.67 15.68 37.95 11.02 21.48 11.02 1.02 7.16 9.32 31.64 12.84 1.00

GLM-4.6v-Flash 55.56 60.08 77.67 62.50 69.92 50.25 51.92 53.00 65.00 65.64 60.23 22.57

Intern-VL-3.5-8B 49.07 53.30 73.02 50.85 58.58 45.85 30.09 51.42 41.98 53.18 48.85 10.87

MiMo-VL-7B-RL 42.59 61.16 80.70 61.40 71.16 56.40 49.65 55.47 70.35 64.39 57.68 21.44

MiMo-VL-7B-SFT 32.41 57.00 69.86 57.57 66.43 46.86 40.00 49.86 62.86 64.68 54.74 22.83

Qwen2.5-VL-7B 44.44 39.90 71.88 44.79 53.23 38.44 27.29 40.83 34.27 50.30 41.78 7.81

Qwen2.5-VL-72B 47.22 53.82 79.41 61.08 70.39 52.45 58.24 55.69 70.59 63.95 61.37 24.48

Molmo-7B-D 1.85 30.00 80.00 22.50 20.00 22.50 10.00 15.00 0.00 45.58 24.00 0.00

Qwen3-VL-30B-A3B 74.07 55.77 72.88 63.65 72.95 52.82 58.40 60.00 67.12 63.02 60.23 26.34

Qwen3-Dense-32B 80.56 63.88 82.00 66.82 72.00 58.76 61.80 63.65 73.47 66.64 65.27 25.84

Open-Source LMMs (thinking)

MiMo-VL-7B-RL 37.96 60.73 78.17 62.80 73.90 52.93 61.95 55.98 74.63 68.05 64.38 29.82

MiMo-VL-7B-SFT 37.96 51.38 74.62 60.00 67.62 50.00 52.88 53.00 60.62 64.04 56.04 23.29

Qwen3-VL-30B-A3B 80.56 59.24 80.35 68.20 69.83 55.70 55.81 55.06 76.69 66.57 62.86 23.56

Table 12: Details Evaluation results on level-2 result(Base-Score Details)

Model Exec. Rate Code-Level Chart-Level

Color Grid Layout Legend Visual Data Text Type Base LLM LMM

Proprietary

Gemini-3-Pro 97.23 52.32 75.31 86.45 63.33 81.58 62.75 77.16 93.86 70.78 72.21 33.41

Claude-Sonnet-4 90.20 47.17 65.29 55.32 56.51 81.50 54.88 80.52 93.29 63.65 66.45 25.40

GPT-5.2 96.04 58.44 80.83 61.51 58.16 84.65 64.66 83.77 94.52 70.93 75.66 33.03

Seed1.5-VL 60.20 44.39 69.37 52.06 52.80 79.85 52.02 77.21 92.66 61.67 65.45 18.30

Seed1.6-VL 70.00 43.61 69.37 52.25 51.32 79.70 53.11 79.46 92.32 61.77 65.40 17.43

Open-Source LMMs (non-thinking)

LLaVA-OV-Qwen2-7B-SI 2.57 15.35 59.38 53.85 30.79 56.34 22.59 52.70 55.33 38.43 37.40 6.40

LLaVA-OV-Qwen2-7B-OV 1.68 15.37 63.48 38.24 64.56 50.15 18.13 48.81 58.43 39.07 35.44 7.35

DeepSeek-VL-7B 30.46 15.32 42.58 34.44 26.24 52.95 22.36 51.56 68.43 35.16 26.87 4.20

kimi-VL-A3B 50.69 26.22 64.50 40.68 39.93 66.08 34.05 64.97 83.03 47.96 44.43 9.01

Qwen2-VL-7B 24.55 18.72 57.60 41.87 32.73 58.65 25.41 55.07 76.38 41.06 34.82 5.27

Qwen2-VL-72B 57.23 27.82 64.07 42.86 40.59 69.53 36.83 64.35 84.76 49.55 39.88 10.02

InternVL-2.5-8B 24.06 23.86 62.25 41.21 36.70 65.03 32.72 62.46 81.14 46.20 41.14 7.47

InternVL-2.5-38B 30.30 30.40 68.25 50.34 48.97 72.92 37.38 70.78 87.96 53.48 52.90 10.67

InternVL-3-8B 4.55 23.22 62.68 57.03 31.34 66.79 29.34 58.63 84.49 46.61 38.02 5.48

InternVL-3-38B 69.80 35.87 68.36 48.66 49.30 75.86 45.33 72.28 90.53 56.74 56.98 13.32

GLM-4V-9B 10.50 19.71 61.79 48.11 34.54 59.98 26.53 57.80 65.80 42.05 34.78 3.87

GLM-4.6v-flash 18.22 48.06 69.23 50.13 51.20 81.56 54.78 77.90 91.86 62.76 63.25 17.91

Intern-VL-3.5-8B 37.52 29.55 68.21 43.87 41.56 69.85 39.58 67.41 83.82 51.30 49.52 9.71

InternVL-3.5-38B 4.26 44.11 92.54 67.22 60.24 72.35 40.38 76.53 88.56 62.64 57.48 8.28

MiMo-VL-7B-RL 17.23 42.72 70.76 54.30 46.81 74.50 49.23 74.57 89.75 59.42 56.95 14.54

MiMo-VL-7B-SFT 30.40 37.36 67.37 48.40 47.24 75.98 46.53 70.80 88.04 56.56 53.97 15.23

Qwen2.5-VL-7B 31.98 26.87 67.22 45.41 38.75 65.34 37.21 63.68 83.65 49.22 44.35 9.44

Qwen2.5-VL-72B 74.55 41.32 67.44 52.16 50.04 77.74 48.65 74.72 91.12 59.31 49.02 16.69

Molmo-7B-D 0.59 16.53 44.44 41.67 50.00 49.70 29.14 41.57 54.44 37.32 29.92 15.50

Qwen3-VL-30B-A3B 38.91 40.07 70.02 51.69 50.06 76.81 48.01 75.52 92.39 59.26 45.77 17.46

Open-Source LMMs (thinking)

MiMo-VL-7B-RL 26.14 42.47 69.05 48.47 45.96 80.76 50.41 74.80 90.56 59.53 61.04 17.81

MiMo-VL-7B-SFT 30.50 40.56 68.22 48.10 46.65 78.42 48.68 73.48 89.86 58.32 59.33 16.38

Qwen3-VL-30B-A3B 13.76 44.00 65.02 48.92 49.19 78.19 50.18 76.18 89.60 59.55 63.04 14.37

Qwen3-32B 63.76 42.90 71.98 50.76 52.24 77.96 52.35 77.43 92.09 61.30 64.15 18.27

Table 13: Details Evaluation results on level-2 result(LLM-Scores Details)

Model Exec. Rate Code-Level Chart-Level

Color Grid Layout Legend Visual Data Text Type Base LLM LMM

Proprietary

Gemini-3-Pro 97.23 85.21 89.26 86.91 78.32 69.72 48.24 73.06 87.00 70.78 72.21 33.41

Claude-Sonnet-4 90.20 79.15 81.38 81.27 75.23 61.89 31.96 68.13 75.81 63.65 66.45 25.40

GPT-5.2 96.04 86.34 90.05 87.25 80.06 71.65 52.00 74.57 89.89 70.93 75.66 33.03

Seed1.5-VL 60.20 78.37 80.30 79.64 72.17 60.46 31.42 67.87 75.55 61.67 65.45 18.30

Seed1.6-VL 70.00 78.48 80.57 80.88 71.56 61.88 30.12 66.99 74.90 61.77 65.40 17.43

Open-Source LMMs (non-thinking)

LLaVA-OV-Qwen2-7B-SI 2.57 47.88 66.73 46.15 44.04 33.85 11.92 40.58 23.08 38.43 37.40 6.40

LLaVA-OV-Qwen2-7B-OV 1.68 42.94 60.59 38.82 68.82 25.88 10.00 33.24 21.18 39.07 35.44 7.35

DeepSeek-VL-7B 30.46 36.13 52.15 34.04 36.15 23.12 2.83 33.42 11.85 35.16 26.87 4.20

kimi-VL-A3B 50.69 56.89 70.88 57.94 52.95 39.54 13.04 52.48 32.36 47.96 44.43 9.01

Qwen2-VL-7B 24.55 46.13 64.17 44.53 45.61 29.86 6.30 42.09 18.50 41.06 34.82 5.27

Qwen2-VL-72B 57.23 64.54 71.67 68.41 56.92 44.60 15.20 55.77 43.17 49.55 39.88 10.02

InternVL-2.5-8B 24.06 55.87 68.90 53.80 49.46 35.35 8.97 50.23 25.70 46.20 41.14 7.47

InternVL-2.5-38B 30.30 65.96 75.74 70.42 65.08 48.55 13.48 59.56 50.74 53.48 52.90 10.67

InternVL-3-8B 4.55 47.98 65.32 44.68 51.06 30.74 6.60 41.70 37.55 46.61 38.02 5.48

InternVL-3-38B 69.80 71.81 76.30 74.05 66.20 51.89 19.55 61.08 57.55 56.74 56.98 13.32

InternVL-3.5-38B 4.26 68.84 82.79 71.40 73.95 51.51 15.35 68.37 58.37 62.64 57.48 8.28

GLM-4V-9B 10.50 49.20 64.43 42.08 40.80 29.76 6.46 41.37 18.07 42.05 34.78 3.87

GLM-4.6v-flash 18.22 76.55 81.66 76.66 71.58 57.31 27.72 66.74 70.03 62.76 63.25 17.91

Intern-VL-3.5-8B 37.52 63.75 74.44 62.32 60.65 43.82 16.29 56.16 40.38 51.30 49.52 9.71

MiMo-VL-7B-RL 17.23 70.29 79.54 69.17 60.46 52.39 26.84 61.67 52.07 59.42 56.95 14.54

MiMo-VL-7B-SFT 30.40 69.11 74.66 66.56 61.03 49.00 22.66 58.90 49.59 56.56 53.97 15.23

Qwen2.5-VL-7B 31.98 55.09 70.80 57.89 54.77 39.49 12.23 50.76 35.12 49.22 44.35 9.44

Qwen2.5-VL-72B 74.55 75.34 76.72 79.06 68.19 56.79 26.03 65.49 66.16 59.31 49.02 16.69

Molmo-7B-D 0.59 38.33 55.00 40.00 60.00 25.00 0.83 30.83 10.00 37.32 29.92 15.50

Qwen3-VL-30B-A3B 38.91 74.04 80.26 78.01 66.96 56.33 25.10 64.51 59.31 59.26 45.77 17.46

Qwen3-VL-32B 63.76 75.96 82.61 78.22 70.49 58.98 30.53 67.80 70.47 61.30 64.15 18.27

Open-Source LMMs (thinking)

MiMo-VL-7B-RL 26.14 76.98 78.08 75.50 66.06 57.53 25.57 63.68 66.81 59.53 61.04 17.81

MiMo-VL-7B-SFT 30.50 75.96 78.18 74.14 65.65 54.04 23.84 61.16 62.49 58.32 59.33 16.38

Qwen3-VL-30B-A3B 13.76 74.93 76.80 78.17 72.12 57.68 26.55 66.14 76.51 59.55 63.04 14.37

Table 14: Details Evaluation results on level-3 details result(Base-Score Details)

Model Exec. Rate Code-Level Chart-Level

Color Grid Layout Legend Visual Data Text Type Base LLM LMM

Proprietary

Gemini-3-Pro 30.03 56.29 83.71 92.55 68.39 84.00 62.26 80.45 96.60 74.28 77.90 35.97

Claude-Sonnet-4 46.65 35.43 68.97 84.08 61.77 74.44 46.73 67.77 89.95 61.13 62.04 16.60

GPT-5.2 20.13 34.46 82.14 80.87 52.64 78.34 43.56 76.09 93.81 61.99 68.40 16.29

Seed1.5-VL 10.86 44.55 76.89 93.94 69.30 73.52 51.75 73.57 95.96 67.58 75.59 22.85

Seed1.6-VL 26.52 44.26 82.28 95.48 74.73 75.42 52.62 77.82 86.47 68.60 76.35 25.77

Table 15: Details Evaluation results on level-3 detail result(LLM-Score Details)

Model Exec. Rate Code-Level Chart-Level

Color Grid Layout Legend Visual Data Text Type Base LLM LMM

Proprietary

Gemini-3-Pro 30.03 82.91 91.70 83.33 86.65 78.69 54.68 74.70 88.72 74.28 77.90 35.97

Claude-Sonnet-4 46.65 65.68 78.97 71.95 78.97 62.71 32.16 57.78 74.32 61.13 62.04 16.60

GPT-5.2 20.13 70.00 84.60 78.10 81.35 68.10 42.51 69.10 77.78 61.99 68.40 16.29

Seed1.5-VL 10.86 77.65 87.06 81.74 90.88 72.94 57.06 69.79 84.12 67.58 75.59 22.85

Seed1.6-VL 26.52 78.77 88.07 86.45 88.55 75.43 52.77 77.12 84.82 68.60 76.35 25.77

Discrepancy Between base-Score and LLM-Score.

Since the Chart-Level LMM-Score serves as the closest proxy to human evaluation (reflecting the actual rendered reality), we re-examine the results in Tab. 8 and Tab. 9. While both the rule-based Base-Score and the semantic-aware LLM-Score at the Code-Level demonstrate a degree of correlation with visual similarity—effectively capturing the model’s generative intent—neither metric successfully approximates the lower, more realistic values of the LMM-Score. For instance, Gemini-3-pro achieves impressive Code-Level scores (Base: 94.44% Type; LLM: 73.75% Type), yet its Chart-Level LMM score remains significantly lower at 40.72%. This consistent disparity highlights a critical semantic modality gap between the textual code domain and the visual chart domain. Code-Level metrics primarily evaluate the syntactic correctness and logical structure of the generation script (i.e., the instructions), but they fail to account for execution-time artifacts, such as occlusion, poor scaling, or color clashing, which only manifest during the rendering process. Fig. 6 illustrates the similarity between Base-Score and LLM-Score across different evaluation dimensions. Both metrics adopt the same weighting scheme, where the data-parameter and color dimensions are assigned higher weights of 0.2, while all other dimensions are weighted at 0.1. Consequently, high Code-Level scores imply that models are proficient at writing valid instructions ("code intent"), but the substantial drop in Chart-Level scores reveals that accurate syntax does not guarantee high-fidelity visual alignment ("visual reality"). However, as shown in Fig. 7, the code-level metrics can still, to some extent, reflect the overall trend of visual fidelity.

Figure 6: Correlation Analysis on Level-1 DR: Base-Score vs. LLM-Score (Code-Level Metrics).

Figure 7: Correlation between Code-level and Chart-level on Level-1 DR

Figure 8: Correlation Analysis on Level-1 CRD: Base-Score vs. LLM-Score (Code-Level Metrics).

Figure 9: Correlation between Code-level and Chart-level on Level-1 CRD

Figure 10: Correlation Analysis on Level-1 CFD: Base-Score vs. LLM-Score (Code-Level Metrics).

Figure 11: Correlation between Code-level and Chart-level on Level-1 CFD

Discrepancy Between LLM-Score and LMM-Score.

Fig.38 illustrates model performance across ten representative task cases, evaluated by both LLM-score for code quality (left) and LMM-score for rendered chart fidelity (right). A clear discrepancy emerges: proprietary models such as GPT-5.2, Gemini-3-Pro, and Claude-Sonnet-4 achieve consistently high LLM-scores across most tasks (often ≥0.2), indicating strong code-level compliance. However, their corresponding LMM-scores are much lower (typically ≤0.35), suggests that code-level evaluation by LLMs lacks precision and tends to yield inflated scores, whereas chart-level assessments align more closely with human visual perception. This contrast highlights that although current models achieve high scores at the code level, they face fundamental limitations in attaining pixel-level chart fidelity, particularly in diverse and challenging task scenarios. Open-source models, in contrast, underperform on both metrics, with particularly low LMM-scores across all tasks. This reveals, on one hand, that relying solely on code-level evaluation leads to a severe text-visual informational discrepancy that fails to accurately reflect actual performance; on the other hand, it underscores a significant performance gap between open-source and proprietary models in actual chart visual fidelity.

12.2 Correlation Between Code Generation Quality and Visual Fidelity

To validate the effectiveness of our evaluation framework, we analyzed the correlation between Code-Level Metrics (assessed via both Base-Score and LLM-Score) and the final Visual Fidelity of the reproduced charts, using the LMM-Score as the ground truth benchmark.

Positive Correlation Between Code Quality and Visual fidelity. A cross-comparison of the Base-Score (Tab. 12) and LLM-Score (Tab. 13) against the LMM-Score reveals a strong positive correlation, confirming that code-level metrics serve as reliable proxies for visual fidelity.

Models that excelled in generating structurally and semantically correct code consistently achieved higher visual fidelity scores. For instance, proprietary models such as Gemini-3-pro and GPT-5.2 dominated the code-level evaluations, achieving high scores in critical dimensions like Grid and Text across both scoring methods (e.g., Gemini-3-pro: Base-Score Text 93.59, LLM-Score Text 84.86). This code proficiency directly translated into the highest LMM-Scores in the benchmark (45.42 and 43.73, respectively).

Conversely, models with weaker code generation capabilities, such as the LLaVA series, exhibited low scores across all code dimensions and correspondingly failed to produce visually coherent charts (LMM-Scores < 3.0). This trend substantiates the hypothesis that high-quality code generation is a prerequisite for, and a strong predictor of, high visual fidelity.

Sensitivity of LLM-Score in Reflecting Visual Nuances. While both code-level metrics correlate with visual outcomes, the LLM-Score demonstrates higher sensitivity to visual defects than the rule-based Base-Score, offering a more accurate reflection of the "visual logic" embedded in the code.

Correction of Score Inflation: The Base-Score often awards high marks based on rigid keyword matching, potentially masking visual alignment issues. For example, Claude-Sonnet-4 achieved a high Base-Score of 85.66 in the Text dimension. However, its LLM-Score for Text dropped to 70.69, aligning more closely with its moderate LMM-Score of 32.36. This suggests that the LLM-based evaluator effectively penalizes code that is syntactically correct but visually suboptimal (e.g., resulting in overlapping text or misaligned labels), thereby reducing the "false positive" rate of rule-based evaluation.

Dimensional Impact: The drop in LLM-Scores for dimensions like Layout and Grid in mid-tier models (e.g., Seed1.5-VL dropping from a Base Layout of 76.16 to an LLM Layout of 56.46) mirrors the steep decline seen in their LMM-Scores. This indicates that the LLM-Score captures the structural integrity of the chart more effectively, serving as a tighter upper bound for potential visual performance.

The "Thinking" Mechanism and Structural Fidelity. The data from "Thinking" models further reinforces the link between code logic and visual fidelity. The introduction of reasoning capabilities (e.g., in MiMo-VL-7B-RL) resulted in significant improvements in structural code dimensions such as Grid (Base-Score improving from 61.70 in the non-thinking variant to 74.48). This enhancement in code structure directly contributed to a notable increase in the LMM-Score (rising from 21.43 to 30.04). This finding suggests that improving a model’s ability to reason about code logic—specifically in defining coordinate systems and layouts—is a critical pathway to bridging the gap between code generation and pixel-perfect visual reproduction.

Conclusion In summary, our experiments demonstrate that code-level metrics are not merely syntactic checks but act as significant indicators of visual fidelity. While the code-visual gap remains, particularly for open-source models, the alignment between robust code scores (especially LLM-Scores) and high LMM-Scores confirms that optimizing for code-level accuracy is essential for achieving high-fidelity chart reproduction.

12.3 Evaluation Case

We present a set of representative case studies generated by Claude Sonnet 4 on the DR task to provide a fine-grained analysis of our multi-dimensional evaluation framework, as shown in Fig. 11–Fig. 16 and Tab. 17–Tab. 19. These cases are carefully selected to illustrate how different evaluation paradigms assess the same chart instance from complementary perspectives.

Specifically, our framework consists of three distinct evaluation protocols: (1) a rule-based method (Base) that quantifies structural and parameter-level consistency directly from code, (2) an LLM-based evaluator (LLM) that performs semantic-level assessment over the generated code, and (3) an LMM-based evaluator (LMM) that measures visual similarity directly at the rendered chart level. For each case, we report not only the final scores from these three evaluators but also detailed sub-dimension scores and corresponding explanations, providing a transparent view of how discrepancies arise.

As illustrated in the example in Fig. 11–12 and Tab. 17, although Base and LLM exhibit relatively consistent scoring trends—indicating strong agreement at the code and structural levels—they both show noticeable divergence from LMM, which evaluates pixel-level visual fidelity. This discrepancy highlights that code-level correctness, even when semantically validated by LLMs, does not necessarily translate into high visual similarity.

Across multiple cases, we consistently observe that the three evaluation methods exhibit a certain degree of correlation, particularly in capturing coarse-grained model ranking. However, the differences in their scoring behaviors reveal complementary strengths: rule-based and LLM-based methods are more sensitive to structural and logical correctness, while LMM-based evaluation better captures perceptual and rendering-level discrepancies such as layout shifts, axis misalignment, and visual styling differences.

Overall, these case studies demonstrate that our multi-dimensional evaluation framework provides a more comprehensive and interpretable assessment by jointly considering code-level correctness and chart-level visual fidelity. This further suggests that relying solely on rule-based or code-based evaluation may lead to incomplete conclusions, whereas incorporating visual-level assessment is crucial for faithfully measuring real-world chart generation quality.

Evaluation Cases

Figure 12: GT_Figure (Left).

Figure 13: Generation_Figure (Right).

Table 16: Detailed multi-dimensional evaluation statistics. By pairing related metrics, we observe comprehensive performance across API, Base, and standard Evaluation reports.

Metric 1 (Score) Metric 2 (Score) Combined Reason / Key Insight

\cellcolorgray!15LLM Report (Overall Score: 38.50)

Chart Type (0.0) Data Sim (0.0) GT is normalized radial radar (radius [0.1,0.9]); GEN plots raw absolute values (0-100). Fundamentally different data mapping.

Visual Params (30.0) Color (80.0) Similar basic colors used, but GEN uses thicker lines, different markers (’o’), and varying radius scaling.

Layout (50.0) Grid (40.0) Figure size, theta offset, and limits differ. GT uses subtle dashed grey grid without y-ticks; GEN shows visible default radial ticks.

Legend (60.0) Text Content (45.0) Legend locations differ (bottom vs. upper-right). Numeric annotation formats, offsets, and tick labels differ significantly.

\cellcolorgray!15Base Report (Overall Weighted Score: 45.84)

Type F1 (1.00) Layout F1 (1.00) Perfect match in basic layout container and chart type detection.

Legend F1 (0.78) Text F1 (0.78) High similarity in legend content and overall text OCR results.

Visual Param (0.74) Data Param (0.14) Visual parameters closely match, but substantial errors exist in data extraction and mapping.

Color F1 (0.00) Grid F1 (0.00) Complete mismatch in color precision and grid configuration.

\cellcolorgray!15LMM Report (Final Similarity Score: 18.0)

Chart Type (100) Color Style (80) Both use radar charts. Colors are semantically consistent (red/green/blue/purple) with only minor hue/opacity differences.

Data (25) Layout (30) Major Errors: Axis alignment/mapping differs drastically; numeric labels do not match GT. Legend moved, title changed, and axis orientation rotated.

Summary: Wharts share the same radar chart type and overall high/low ordering by series, but there are multiple clear visual mismatches: major data-axis alignment/annotation differences and substantial layout/annotation shifts (legend, ticks, orientation), plus minor color/style and numeric-label placement differences.

Evaluation Cases

Figure 14: GT_Figure (Left).

Figure 15: Generation_Figure (Right).

Table 17: Detailed multi-dimensional evaluation statistics. By pairing related metrics, we observe comprehensive performance across API, Base, and standard Evaluation reports.

Metric 1 (Score) Metric 2 (Score) Combined Reason / Key Insight

\cellcolorgray!15LLM Report (Overall Score: 85.00)

Chart Type (100.0) Data Sim (65.0) Both render side-by-side grouped vertical bar charts. Most values match, but several numeric differences exist (e.g., Microsoft 212 vs 218), producing perceptible height differences.

Visual Params (75.0) Color (100.0) Same color arrays used. Exact bar and figure sizes differ (13x8 vs 14x6), which slightly changes spacing and relative bar thickness.

Layout (80.0) Grid (85.0) Both have 1x2 subplots with horizontal y-axis grids. GT’s grid is dashed and more prominent; placement methods for legends cause slight margin differences.

Legend (85.0) Text Content (95.0) Legends are centered above plots with matching entries. Titles and labels match exactly; GT explicitly sets larger font sizes.

\cellcolorgray!15Base Report (Overall Weighted Score: 77.67)

Type F1 (1.00) Layout F1 (1.00) Perfect structural match in basic layout container and chart type detection.

Data Param (0.99) Color F1 (0.96) Exceptional performance in extracting data values and precise color matching.

Text F1 (0.88) Visual Param (0.85) Strong alignment in text OCR results and visual parameter rendering.

Grid F1 (0.12) Legend F1 (0.00) Model struggles significantly with extracting legend configurations and grid properties.

\cellcolorgray!15LMM Report (Final Similarity Score: 85.0)

Chart Type (100) Color Style (80) Both are grouped vertical bar charts. Color hues are consistent; minor style differences exist in stroke thickness and bar edges.

Data (95) Layout (80) Heights, ordering, and trends match with negligible pixel differences. Minor shifts in font sizes, legend marker spacing, and subplot margins.

Summary: Charts are visually very similar with matching chart types, data trends, axes, and layout; only minor differences in styling (font weight/size, slight legend marker spacing) and extremely small numeric/text rendering variations are present.

is a direct indicator of multimodal interaction [16, 54, 83], it can reveal whether a text token successfully grounds in a relevant visual region [82, 93]. Therefore, by visualizing cross-attention maps for object-related words of Qwen2.5- VL [2] in Fig. 2, we aim to uncover alignment patterns and weaknesses not apparent from standard accuracy metrics. Interestingly, our cross-attention analysis reveals a systematic discrepancy: Attribute words [27, 36, 56] produce sharper and more localized activations in the visual modality, while object nouns [52] result in diffuse and scattered attention patterns. We attribute this discrepancy to biases in semantic reference. In large-scale multimodal corpora, attribute words, such as colors or textures, correspond to specific and spatially localized visual patterns, while object nouns occur in diverse contexts, diluting their spatial association. Moreover, attribute words naturally map to low-level visual features, while object nouns rely on high-level semantic representations often varied across instances, which leads to poor alignment without explicit supervision. This finding suggests that improving fine-grained object understanding requires explicitly strengthening crossmodal correspondence for object nouns, and thus inspires us to pursue direct supervision between object words and their associated visual regions. To provide such supervision signals, an enriched video understanding dataset that pairs object-level visual annotations with natural language prompts containing clear references is required. Thanks to earlier visual-prompt-based approaches [50, 89], collecting training data with mask annotations is not difficult. We start from VideoRefer [89], a video fine-grained object understanding dataset providing frame-level object masks aligned with textual prompts via placeholder tokens used for visual prompting. While these masks are valuable, the associated text does not contain clear natural language references to the objects. We therefore design a GPT-4o-powered [49] data refinement pipeline and construct the NL-Refer dataset. Specifically, for each placeholder, we automatically replace it with a concise natural language description of the specific instance, informed by the context. Based on the NL-Refer dataset, we propose SWIM (See What I Mean), a simple yet effective training strategy that explicitly aligns vision and language representations to strengthen fine-grained object understanding in MLLMs. Specifically, during supervised fine-tuning, SWIM extracts cross-attention maps for object nouns from multiple intermediate layers and aligns them with ground-truth object masks, enforcing spatial consistency between textual identity and visual grounding. By providing explicit alignment signal throughout training, SWIM guides the model to preserve and utilize fine-grained object-level information, enabling more precise visual localization from purely textual prompts at inference. Extensive experiments across finegrained object understanding benchmarks demonstrate that SWIM enhances text–visual alignment and outperforms visual-prompt-dependent approaches. We summarize our contributions as follows. • We point out design limitations in existing fine-grained object understanding models and the insufficient vision–language alignment in general MLLMs. Based on the observed systematic discrepancy, we introduce NLRefer dataset, in which each object is referred with explicit natural language expressions. • We propose a novel training strategy, SWIM (See What I Mean), which explicitly enforces alignment between visual content and object nouns during training, resulting in a model that requires no visual prompt inputs at inference. • Through experiments on fine-grained object understanding benchmarks, SWIM demonstrates consistent improvements over visual-prompt-based approaches. Quantitative and qualitative analyses of text–visual alignment further corroborate our claim. Evaluation Cases [Uncaptioned image] Figure 16: GT_Figure (Left). [Uncaptioned image] Figure 17: Generation_Figure (Right). Table 18: Detailed multi-dimensional evaluation statistics (Combination). By pairing related metrics, we observe comprehensive performance across API, Base, and standard Evaluation reports. Metric 1 (Score) Metric 2 (Score) Combined Reason / Key Insight \cellcolorgray!15LLM Report (Overall Score: 39.50) Chart Type (20.0) Data Sim (25.0) GT uses deterministic analytic plots and specific categorical values; GEN samples random data (np.random) and uses histogram-based density plots, completely altering the geometry and underlying distributions. Visual Params (40.0) Color (30.0) GT applies specific bar widths, lw=2 overlays, and a light color palette (e.g., lightgreen); GEN uses default histogram styles and a mismatched green/blue/orange palette. Layout (50.0) Grid (70.0) GT arranges a 2x4 grid (17x10) with specific polar dashed grids; GEN uses a 3x4 gridspec (16x12) with default Cartesian histogram grids, causing noticeable aspect differences. Legend (60.0) Text Content (45.0) GT features precise legend locations and annotation texts (exact percentages); GEN uses default placements and simulates a word cloud with different text. \cellcolorgray!15Base Report (Overall Weighted Score: 46.02) Text F1 (83.92) Type F1 (75.00) Strong performance in text OCR and reasonable detection of the multiple basic chart types present. Legend F1 (71.50) Data Param (62.10) Moderate alignment in legend extraction and data parameter identification, though overall recall remains insufficient. Visual Param (54.71) Color F1 (25.41) Model struggles significantly with visual rendering parameters and shows severe color mismatches. Layout F1 (0.00) Grid F1 (0.00) Complete failure in capturing the complex multi-subplot arrangement and grid configurations. \cellcolorgray!15LMM Report (Final Similarity Score: 0.0) Chart Type (100) Color Style (0) Successfully identifies the complex chart mix. However, Major Errors exist: missing KDE lines, hatching, bar outlines, and severe donut/wordcloud color mismatches. Data (0) Layout (0) Major Errors: Fundamental data deviations (donut rings collapsed from 90%/40% to a single 50% ring). Subplot composition, aspect ratios, and KDE curve presence are entirely altered. Summary: Automatic failure (0 score) triggered by multiple major errors. The model fails fundamentally in data presentation (e.g., random data sampling instead of deterministic plotting, collapsed donut rings) and layout composition (missing KDE curves, altered grid arrangement).

12.4 Error Case Study:

To improve interpretability, we conduct a fine-grained analysis of model failures across the three hierarchical levels of our benchmark. Building upon the preliminary observations in Section 4.2.2 and Appendix 9, we further perform a granular metadata-based study on failure cases to refine our taxonomy and better characterize the dominant error sources at each level. Selected qualitative examples are provided in following pages. Fig. 18–Fig. 31 show the common error types in three tasks. These include color consistency errors, component position errors, data extraction and encoding errors, and axis scale errors, among others. These error types are shared across the three tasks and constitute the main errors related to the model’s visual perception and chart code reconstruction abilities.

Level 1: Perception and Data Extraction Failures. At Level 1, failures are primarily attributable to insufficient visual perception and weak data extraction capabilities. Models frequently overlook fine-grained visual attributes—such as grid configurations, marker styles, and line patterns—resulting in degraded visual fidelity. In addition, inaccurate or incomplete data extraction, particularly in chart-from-data (CFD) tasks, leads to erroneous or missing plotted values. Although the generated code is often syntactically correct and executable, the rendered charts fail to faithfully reproduce the target visualization.

Level 2: Reasoning and Instruction Alignment Failures. While Level 2 inherits the perception and extraction limitations observed in Level 1, the dominant bottleneck shifts to reasoning capacity. The editing tasks at this level involve substantially more complex and compositional instructions. Limited reasoning ability prevents models from executing valid edits that simultaneously respect the original visual context and satisfy the specified transformation requirements. This explains the consistent performance gap between Level 2 and Level 1 across evaluated models.

Level 3: Long-Context and Instruction Comprehension Failures. The severe performance degradation observed at Level 3 is mainly driven by long-context processing limitations. Handling extensive raw tabular data consumes a substantial portion of the model’s available context window, leading to the well-documented “lost-in-the-middle” phenomenon. Consequently, models exhibit incomplete data extraction and weakened understanding of plotting instructions. This results in a pronounced decline in both execution success rate and visual fidelity.

Common Error Category: Color Errors

Case 1: Box Chart (level3)

(a) Ground Truth

(b) Model Output

Analysis: Although the model identified the color sequence in the legend area, this perception failed to transfer to the plot objects within the coordinate system, reflecting a logical inconsistency between local feature recognition and global mapping.

Case 2: Heatmap Chart (level1)

(c) Ground Truth

(d) Model Output

Analysis: Significant discrepancies in color transitions are observed between the model-generated output (right) and the ground-truth (GT) figure, indicating limitations in the model’s ability to accurately perceive and reproduce subtle variations in color.

Case 3: Combination Chart (level2)

(e) Ground Truth

(f) Model Output

Analysis: The legend colors in the model output on the right do not match those in the GT Figure, and there is a noticeable color discrepancy at X=18. This indicates that the model lacks fine-grained image perception capabilities in complex charts.

Figure 18: Qualitative Error Analysis. Color inconsistency issues across different chart types, including mismatches among axis colors, legend entries, and the main plot elements.

Common Error Category: Color Errors

Case 1: Box Chart(level1)

(a) Ground Truth

(b) Model Output

Analysis: The color of the bottom subplot in the model output on the right does not match the GT Figure; the model failed to understand the color-coded regions in the area chart.

Case 2: Heatmap Chart(level2)

(c) Ground Truth

(d) Model Output

Analysis: The color filling in the model output on the right is inconsistent with the GT Figure and lacks visual appeal; the transparency of the filled colors in the area regions has not been appropriately adjusted.

Case 3: Combination Chart(level1)

(e) Ground Truth

(f) Model Output

Analysis: The color filling in the model output on the right is completely opposite to that of the GT Figure, indicating the model’s inability to accurately recognize colors. Additionally, there are issues with the recognition of the legend.

Figure 19: Qualitative Error Analysis. Illustration of color consistency errors across various chart types, highlighting mismatches between axes, legend labels, and primary visual elements.

Common Error Category: Component Position Errors

Case 1: Bar Chart (level1)

(a) Ground Truth

(b) Model Output

Analysis: The legend position in the model output on the right does not match that of the GT Figure, and the placement of the numerical values is also completely different.

Case 2: Bar Chart (level1)

(c) Ground Truth

(d) Model Output

Analysis: The legend position in the model output on the right does not match that of the GT Figure, severely obscuring the data portion of the bar chart and causing significant visibility issues.

Case 3: Combination Chart (level2)

(e) Ground Truth

(f) Model Output

Analysis: The subplot position in the model output on the right does not match that of the GT Figure, severely obscuring the data portion of the error points and causing significant visibility issues.

Figure 20: Qualitative Error Analysis. Common positional mistakes in model outputs, including overlap between the legend and chart elements, overlapping visual elements within the chart, and text overlapping with graphical components.

Common Error Category: Component Position Errors

Case 1: Errorpoint Chart (level2)

(a) Ground Truth

(b) Model Output

Analysis: The legend position in the model output on the right does not match that of the GT Figure, severely obscuring the chart title and making the letters in the title illegible.

Case 2: Pie Chart (level3)

(c) Ground Truth

(d) Model Output

Analysis: The model failed to accurately extract the absolute spatial coordinates of the legend, leading to positional displacement and visual overlapping with the title, reflecting a failure in spatial perception for complex layouts.

Case 3: Combination Chart(level3)

(e) Ground Truth

(f) Model Output

Analysis: The model exhibits a clear visual perception bottleneck in spatial reasoning, failing to recognize the physical boundaries between the legend, title, and chart body. This leads to severe visual overlapping and positional misalignment, as the model overlooks the layout constraints required to prevent component occlusion.

Figure 21: Qualitative Error Analysis. Examples of frequent positional inconsistencies in model-generated charts, such as legend–plot overlap, internal element collisions, and text–graphic occlusion.

Common Error Category: Data Encoding Errors

Case 1: Combination Chart (level2)

(a) Ground Truth

(b) Model Output

Analysis: The data distribution in the model output on the right is inconsistent with that of the GT Figure, indicating insufficient data accuracy and a deficiency in the model’s ability to recognize and analyze the original chart’s data.

Case 2: Heatmap Chart (level3)

(c) Ground Truth

(d) Model Output

Analysis: In the generated heatmap, the model only reproduced the data for the "Total" indicator, completely omitting all other component indicators (e.g., S1, S2, C3, E3, etc.), which reflects a critical failure in data extraction and comprehensive information reproduction capabilities.

Case 3: Combination Chart (level3)

(e) Ground Truth

(f) Model Output

Analysis: The model made critical errors in data extraction, failing to accurately reproduce the histogram bar heights, KDE curve shape, and X-axis range from the ground truth, resulting in a fundamentally incorrect data distribution.

Figure 22: Qualitative Error Analysis. Model outputs exhibiting data-related inaccuracies, such as incorrect data extraction and erroneous data distributions, leading to poor visual fidelity.

Error Category: Data Encoding Errors

Case 1: Radar Chart (level1)

(a) Ground Truth

(b) Model Output

Analysis: The data distribution in the model output on the right is inconsistent with that of the GT Figure, indicating insufficient data accuracy and a deficiency in the model’s ability to recognize and analyze the original chart’s data.

Case 2: Combination Chart (level1)

(c) Ground Truth

(d) Model Output

Analysis: The data distribution in the model output on the right is inconsistent with that of the GT Figure, indicating insufficient data accuracy and a deficiency in the model’s ability to recognize and analyze the original chart’s data.

Case 3: Combination Chart (level1)

(e) Ground Truth

(f) Model Output

Analysis: The data distribution in the model output on the right is inconsistent with that of the GT Figure, indicating insufficient data accuracy and a deficiency in the model’s ability to recognize and analyze the original chart’s data.

Figure 23: Qualitative Error Analysis. Examples of data misrepresentation in model-generated charts, including extraction mistakes and incorrect value distributions, resulting in degraded visual fidelity.

Common Error Category: Axis Scale Errors

Case 1: Bar Chart (level1)

(a) Ground Truth

(b) Model Output

Analysis: In the model-generated chart, the y-axis range is incorrectly set to 0–3.5, which is narrower than the 0–4 range in the ground truth, resulting in the upper portion of the stacked area chart being truncated. Additionally, the leftmost segment of the chart is missing. These issues indicate deficiencies in the model’s data extraction capability and its ability to accurately infer the underlying numerical scale.

Case 2: Heatmap Chart (level2)

(c) Ground Truth

(d) Model Output

Analysis: In the generated chart, the Y-axis range of the left subplot is incorrectly compressed to 0.53–0.76, and the legend is misplaced over the data area, both of which severely distort the data representation and visual clarity compared to the ground truth.

Case 3: Combination Chart (level2)

(e) Ground Truth

(f) Model Output

Analysis: The y-axis range in the model output on the right is problematic, causing part of the data to be invisible. This indicates an insufficient match between the axis range and the data. Additionally, errors occurred in data recognition and analysis.

Figure 24: Qualitative Error Analysis. Scaling inconsistencies in the model outputs, leading to issues such as the disappearance of primary chart elements or disproportionate visual layouts, thereby reducing overall visual fidelity.

Common Error Category: Type Errors

Case 1: Combination Chart (level1)

(a) Ground Truth

(b) Model Output

Analysis: The chart type in the model output on the right is inconsistent with that of the GT Figure; the scatter plot has been misidentified as a contour plot.

Case 2: Bar Chart (level1)

(c) Ground Truth

(d) Model Output

Analysis: The chart type in the model output on the right is inconsistent with the GT Figure; the regular bar chart has been misidentified as a stacked bar chart, which deviates significantly from the original.

Case 3: Combination Chart (level2)

(e) Ground Truth

(f) Model Output

Analysis: The chart type in the model output on the right is inconsistent with the GT Figure; the bar chart on the right differs significantly from the stacked bar chart in the GT Figure.

Figure 25: Qualitative Error Analysis. Examples of chart-type misclassification in generated results, including confusion between stacked and standard bar charts, horizontal and vertical orientations, and other structural type mismatches.

Common Error Category: Type Errors

Case 1: Combination Chart (level2)

(a) Ground Truth

(b) Model Output

Analysis: The chart type in the model output on the right is inconsistent with that of the GT Figure; the right subplot differs significantly from the corresponding subplot in the GT Figure. The orientation of the bars in the bar chart is inconsistent, and there are notable differences in the remaining parts as well.

Case 2: Violin Chart (level3)

(c) Ground Truth

(d) Model Output

Analysis: The chart type in the model output on the right differs from that of the GT Figure. The GT Figure is a violin chart, but the model generated a box chart.

Case 3: Polar Chart (level3)

(e) Ground Truth

(f) Model Output

Analysis: The chart type in the model output on the right differs from that of the GT Figure. The GT Figure is a circular bar chart, but the model generated a regular bar chart.

Figure 26: Qualitative Error Analysis. Incorrect chart-type predictions in model outputs, such as confusing stacked bar charts with standard bar charts, or misclassifying horizontal and vertical bar charts, among other type-related mistakes.

Common Error Category: Grid Errors

Case 1: Line Chart (level1)

(a) Ground Truth

(b) Model Output

Analysis: The grid line style in the model-generated output (right) is inconsistent with that of the ground-truth (GT) figure, with dashed lines appearing on the right side where they are not present in the reference. This discrepancy indicates a deficiency in the model’s ability to perceive and faithfully reproduce fine-grained visual details.

Case 2: Line Chart (level1)

(c) Ground Truth

(d) Model Output

Analysis: The grid line style in the model output on the right is inconsistent with that of the GT Figure, indicating a deficiency in the model’s ability to perceive fine-grained visual details.

Case 3: Combination Chart (level2)

(e) Ground Truth

(f) Model Output

Analysis: The grid line style in the model output on the right is inconsistent with that of the GT Figure, indicating a deficiency in the model’s ability to perceive fine-grained visual details.

Figure 27: Qualitative Error Analysis. Discrepancies in grid configuration within model-generated charts, including mismatches in horizontal and vertical gridlines, missing or extraneous grids, and incorrect gridline styles or thickness.

Common Error Category: Grid Errors

Case 1: Combination Chart (level2)

(a) Ground Truth

(b) Model Output

Analysis: The grid line style in the model output on the right is inconsistent with that of the GT Figure, indicating a deficiency in the model’s ability to perceive fine-grained visual details.

Case 2: Line Chart (level2)

(c) Ground Truth

(d) Model Output

Analysis: The grid line style in the model output on the right is inconsistent with that of the GT Figure, indicating a deficiency in the model’s ability to perceive fine-grained visual details.

Case 3: Polar Chart (level3)

(e) Ground Truth

(f) Model Output

Analysis: The grid line style in the model output on the right differs from that of the GT Figure. In the original figure, there are no horizontal grid lines. Additionally, there are issues with the data in the chart.

Figure 28: Qualitative Error Analysis. Examples of incorrect grid settings in generated charts, such as improper horizontal/vertical gridlines, grid presence inconsistencies, and errors in line style or thickness.

Common Error Category: Text Occlusion Errors

Case 1: Bar Chart (level2)

(a) Ground Truth

(b) Model Output

Analysis: There is a serious text overlap issue in the model output on the right, with overlapping text between the upper and lower subplots, which severely affects the visual aesthetics.

Case 2: Bar Chart (level2)

(c) Ground Truth

(d) Model Output

Analysis: The legend and title in the model output on the right severely overlap, significantly impacting the visual aesthetics.

Case 3: Combination Chart (level2)

(e) Ground Truth

(f) Model Output

Analysis: There is severe overlap among the labels in the model output on the right, which greatly impacts the visual aesthetics. Additionally, there is a noticeable deficiency in the recognition of numerical values.

Figure 29: Qualitative Error Analysis. Frequent text overlap issues in model-generated charts, occurring across legends, titles, x-axis labels, y-axis labels, and other textual elements.

Common Error Category: Visual Style Errors

Case 1: Combination Chart (level2)

(a) Ground Truth

(b) Model Output

Analysis: The transparency in the left plot of the model output on the right is inconsistent with that of the GT Figure. Additionally, the legend style differs—the GT Figure uses a dashed line, while the model output uses a solid line. Furthermore, the text bolding also deviates from the GT Figure.

Case 2: Combination Chart (level2)

(c) Ground Truth

(d) Model Output

Analysis: The colors in the left plot of the model output on the right are inconsistent with those in the GT Figure, and the legend positions also differ. In the GT Figure, the bar chart on the right uses different colors, whereas in the model output, it is white.

Case 3: Bar Chart (level1)

(e) Ground Truth

(f) Model Output

Analysis: In the generated chart, the dashed grid lines that were present in the ground truth are completely missing, which directly reflects the model’s failure to capture and reproduce fine-grained visual attributes.

Figure 30: Qualitative Error Analysis.Deficiencies in visual styling of model-generated charts, including improper control of line width, transparency, color gradients, edge outlines, and other visual parameters that affect overall appearance.

Common Error Category: Visual Style Errors

Case 1: Bar Chart (level2)

(a) Ground Truth

(b) Model Output

Analysis: In the generated chart, the dashed grid lines present in the ground truth are completely missing, and the color gradients are not accurately replicated. This directly reflects the model’s failure to capture and reproduce fine-grained visual attributes.

Case 2: Line Chart (level3)

(c) Ground Truth

(d) Model Output

Analysis: During the visual perception stage, the model failed to capture key stylistic attributes, overlooking the bolded line width and prominent marker sizes present in the original chart

Case 3: Combination Chart (level3)

(e) Ground Truth

(f) Model Output

Analysis: In the generated chart, the histogram bars lack the distinct black borders that were present in the ground truth, which reduces the visual clarity and contrast of the bar distribution, reflecting the model’s failure to reproduce fine-grained visual details.

Figure 31: Qualitative Error Analysis. Errors in visual parameterization, such as incorrect line thickness, opacity, gradient schemes, and edge styling, leading to suboptimal visual presentation.

Special Error Category for level2

Case 1: Bar Chart (level2)

(a) Ground Truth

(b) Model Output

Analysis: The generated output exhibits a severe logical discrepancy compared to the ground truth. It misrepresents the 100% stacked proportions in the top subplot and completely omits the corresponding negative difference bars in the bottom subplot, failing to reflect the required compositional mathematical constraints.

Case 2: Line Chart (level2)

(c) Ground Truth

(d) Model Output

Analysis: As illustrated, the significantly complex instruction of embedding a box plot inset within a dual-axis line chart exceeds the model’s reasoning capacity; this limitation prevents it from performing valid edits that satisfy the visual context, ultimately resulting in a structurally collapsed inset and severely distorted secondary axis data mapping.

Case 3: Combination Chart (level2)

(e) Ground Truth

(f) Model Output

Analysis: The highly complex compound plotting instructions—requiring an inset chart, a Pareto front line, custom markers, and text annotations—overwhelmed the model’s execution capacity, resulting in unimplemented key features (e.g., missing lines and text labels) and partial data loss (e.g., the categorical X-axis being erroneously replaced by arbitrary numerical values).

Figure 32: Level 2–Specific Errors: Due to the substantially more complex editing instructions, reasoning becomes the primary bottleneck. Limited reasoning capacity prevents the model from producing edits that simultaneously satisfy the visual context and the given instructions, leading to consistently lower performance than Level 1.

Special Error Category for level2

Case 1: Combination Chart (level2)

(a) Ground Truth

(b) Model Output

Analysis: As illustrated by the calibration dashboard, the highly complex composite instructions—which demand cross-plot axis sharing, and data-conditional annotations—overwhelm the model’s logical reasoning and multi-task scheduling capacities; this not only causes a complete collapse of structural layout constraints but also leads to the total omission of crucial analytical visual elements, such as the maximum error highlight.

Case 2: Combination Chart (level2)

(c) Ground Truth

(d) Model Output

Analysis: When instructions simultaneously demand dynamic statistical calculations (e.g., binning for means and standard deviations) and rigorous spatial/layering constraints (e.g., absolute coordinate positioning and multiple Z-order configurations), the model suffers a complete layout collapse due to insufficient spatial reasoning. This is evidenced by the wildly disproportionate histogram that obfuscates the core plotting area, highlighting the model’s limitations in complex, multi-task visual rendering.

Case 3: Combination Chart (level2)

(e) Ground Truth

(f) Model Output

Analysis: The catastrophic layout failures shown in the figure, including the disconnected CDF axis and missing histogram data, highlight a critical reasoning bottleneck. When confronted with highly reasoning-dependent tasks like reformatting the chart into a complex left-right layout with shared axes, the model fails completely, resulting in an unusable output.

Figure 33: Level 2–Specific Errors: Due to the substantially more complex editing instructions, reasoning becomes the primary bottleneck. Limited reasoning capacity prevents the model from producing edits that simultaneously satisfy the visual context and the given instructions, leading to consistently lower performance than Level 1.

Special Error Category for level3

Case 1: Heatmap Chart (level3)

(a) Ground Truth

(b) Model Output

Analysis: As evidenced by the extensive data blanks in the generated heatmap, processing lengthy raw tabular data heavily occupies the context window capacity, inducing the "lost-in-the-middle" phenomenon and ultimately resulting in highly incomplete data extraction.

Case 2: Combination Chart (level3)

(c) Ground Truth

(d) Model Output

Analysis: In the generated bubble chart, the model suffered from severe multi-dimensional mapping failure when processing complex data (X, Y, bubble size, and color), not only erroneously altering the linear axis to a logarithmic scale—causing extreme distortion in data distribution—but also completely losing the continuous color mapping.

Case 3: Combination Chart (level3)

(e) Ground Truth

(f) Model Output

Analysis: The sheer volume of the raw tabular data causes severe information attenuation within the context window, resulting in significant categorical omissions and distorted numerical values in the generated chart.

Figure 34: Level 3–Specific Errors: Performance degradation primarily stems from long-context limitations. Processing extensive tabular data exhausts the context window, leading to incomplete data extraction and weakened instruction comprehension, which significantly reduces execution success rate and visual fidelity.

Special Error Category for level3

Case 1: Combination Chart (level3)

(a) Ground Truth

(b) Model Output

Analysis: The extraordinarily large tabular input coupled with complex plotting instructions caused severe context overload in the model, leading not only to catastrophic data truncation and massive omissions but also to instruction forgetting and complete semantic hallucinations.

Case 2: Combination Chart (level3)

(c) Ground Truth

(d) Model Output

Analysis: As illustrated in the generated heatmap, the complex two-dimensional tabular data exceeds the model’s effective context capacity, triggering a severe ’lost-in-the-middle’ phenomenon that ultimately results in massive row-level data extraction failures and extensive blank areas.

Case 3: Combination Chart (level2)

(e) Ground Truth

(f) Model Output

Analysis: The generated output exhibits a severe structural degradation compared to the ground truth. While the GT presents a complex dual-panel layout simultaneously comparing two distinct demographics (Urban and Rural), the generated chart collapses this into a single, simplified slope graph plotting a derived ratio. This stark contrast highlights a critical failure in processing extensive data context and comprehending complex layout instructions, resulting in incomplete visual extraction.

Figure 35: Level 3–Specific Errors: Performance degradation primarily stems from long-context limitations. Processing extensive tabular data exhausts the context window, leading to incomplete data extraction and weakened instruction comprehension, which significantly reduces execution success rate and visual fidelity.

13 Limitations

While Chart2Code serves as a robust benchmark, we identify two limitations to be addressed in future work:

Linguistic Diversity and Generalization: The dataset annotations and task prompts are exclusively in English. This design choice may overestimate the capabilities of current models when applied to multilingual contexts. Real-world data visualization often involves diverse languages and cultural formatting standards. Consequently, the current benchmark does not fully evaluate how well LMMs can generalize to non-English queries or preserve text rendering accuracy in multi-byte character sets (e.g., CJK languages).

Evaluation Bias in LMM-based Judges: Our evaluation protocol relies on strong LMMs to assess code correctness and visual fidelity. While this ensures scalability, it poses a risk of evaluation bias. LMM-based judges might exhibit leniency towards generated code patterns similar to their own training data, or struggle to accurately penalize subtle visual artifacts (such as layout misalignment) that do not trigger code execution errors. Developing more objective, reference-free visual metrics remains an open challenge.

Figure 36: Human vs model evaluate system.

14 Human Evaluation

14.1 Correlation between Automated Metrics and Human Evaluation

To validate the reliability of our proposed automated metrics and their alignment with human perception, we conducted a human evaluation study involving 20 undergraduate students from the School of Computer Science. Participants assessed both the ground truth (GT) charts and the generated charts across multiple dimensions, aiming for a comprehensive evaluation of the reproduction quality, the evaluation system is illustrated in Fig. 36. Regarding the sampling strategy, we randomly selected 300 result images for the DR task and the Level 2 task, respectively. For the CRD and CFD tasks, we evaluated the complete set of output results. We assessed the outputs generated by seven models: Qwen3-VL-30b_A3B-Instruct, Qwen3-VL-30b_A3B-Think, MiMo-Vl-7B-RL, Gemini-3-pro, InternVL-3-8B, Qwen2.5-VL-7B, and GPT-5.2. The detailed results for each task, along with the calculated correlation coefficients, are presented in Fig. 37(a), Fig. 37(b), Fig. 37(c), Fig. 37(d) .

From the results shown in the Fig. 37(a), Fig. 37(b), Fig. 37(c), Fig. 37(d), we observe that the correlation between LMM scores and human ratings is generally strong across most tasks, with the exception of the CRD setting, where the linear correlation is comparatively weaker. This suggests that LMMs can effectively evaluate the quality differences between two charts, and that their evaluation criteria broadly align with those used by human annotators—leading to similar scoring trends. The higher Spearman and Kendall correlation coefficients further indicate that, rather than achieving precise numerical agreement, the models are more reliable at capturing the relative ranking of chart quality. In other words, although there are notable discrepancies between the absolute LMMs scores and human ratings, LMMs remain capable of distinguishing higher-quality charts from lower-quality ones.

Regarding error metrics, both the RMSE and Bias values reveal a consistent upward deviation of LMM predictions from human evaluations, indicating a tendency for LMMs to overestimate chart quality. This systematic bias highlights an important limitation: while LMM scores are correlated with human assessments, they are not yet calibrated to the same level of strictness as human judgment. Among different models, proprietary models such as GPT-5.2 and Gemini-3-Pro tend to align more closely with the regression trend line—demonstrating smaller deviations from human evaluations—whereas many open-source models exhibit higher variance.

In summary, these findings demonstrate that LMM-based automatic scoring provides a useful, though not perfect, signal for chart-quality evaluation. Despite the remaining challenges in fine-grained assessment for complex chart-understanding tasks, LMM scores still offer sufficiently reliable trends to serve as effective auxiliary indicators in large-scale benchmarking.

(a) Human vs. model evaluation: correlation between human evaluation and LMM Score in DR.

(b) Human vs. model evaluation: correlation between human evaluation and LMM Score in CRD.

(c) Human vs. model evaluation: correlation between human evaluation and LMM Score in CFD.

(d) Human vs. model evaluation: correlation between human evaluation and LMM Score in Level 2.

Figure 37: Human vs. model evaluation across all tasks: correlation between human evaluation and LMM Score.

Figure 38: Analysis of model performance on different task cases with LLM-score and LMM-score.

14.2 Performance Comparison

To provide a more intuitive comparison of LMM performance across our hierarchical tasks, Fig. 40, Fig. 41, Fig. 42, Fig. 43, Fig. 44 presents the Executability Rate, LLM Score, and LMM Score, benchmarked against human performance.

High Executability in Proprietary Models, Approaching Human Levels In terms of Execution Rate (Exec.Rate), proprietary models demonstrate exceptional stability. This indicates that current top-tier LMMs have largely solved the challenge of generating syntactically correct and runnable code for simple chart reproduction.

Visual Fidelity Remains the Major Bottleneck The LMM-Score, which measures the visual similarity between the generated chart and the ground truth, reveals the biggest challenge. Most models show a sharp decline in this metric compared to their Code-Level scores.

The Gap Between Code Generation and Visual Perception. There is a noticeable discrepancy between the blue bars (LLM-Score) and grey bars (LMM-Score). This indicates that while models understand how to write code, they often fail to perceive or reproduce the fine-grained visual nuances (such as exact color hex codes, marker styles, or specific layout constraints) required for pixel-perfect reproduction.

While top-tier models have mastered the syntax required to generate charts (High Exec.Rate), the semantic understanding of the data (LLM-Score) is being revolutionized by "thinking" models. However, achieving high visual fidelity (LMM-Score) remains a difficult frontier for most models.

Figure 39: Correlation of the model performance (i.e, LMM-score) on different manually annotated difficulty levels (i.e., Easy, Medium, Hard) on Level 1, 2, 3, respectively.

Figure 40: Human vs model performance: LLM-score and LMM-score across level 1 direct tasks.

Figure 41: Human vs model performance: LLM-score and LMM-score across level 1 customize tasks.

Figure 42: Human vs model performance: LLM-score and LMM-score across level 1 figure tasks.

Figure 43: Human vs model performance: LLM-score and LMM-score across level 2 tasks.

Figure 44: Human vs model performance: LLM-score and LMM-score across level 3 tasks.

15 Evaluation Metric Details

15.1 Overall

To better evaluate the performance of different models, we conduct comparative assessments from two levels: the code-level and the chart-level. Throughout the evaluation process, we first examine the executability of the generated code. The execution rate is defined as the ratio between the number of executable code snippets that successfully generate images (s) and the total number of tasks (t). Formally, the execution rate is expressed as:

exec_rate=st. (1)

The execution rate is reported as a percentage.

At the code-level, we first extract plotting elements from the matplotlib.Figure object and propose eight evaluation dimensions as the base evaluation. The detailed specifications are given in 15.2. Subsequently, we leverage gpt-5-mini to perform a holistic similarity assessment of the code’s visualization results, thereby providing a more reliable confidence score at the code level. We refer this as LLM-Score.

At the chart-level, we input the executable code into gpt-5-mini for image-based evaluation. By designing specific prompts, the large multimodal model (LMM) assesses multiple dimensions and produces an aggregated score. This chart-level evaluation offers an intuitive similarity measure of the visual outputs, thereby serving as a direct indicator of model performance. We refer this as LMM-Score. The implementation details of these two evaluation mechanisms are described as follows.

15.2 Base Evaluation

To evaluate visualization effects from the code perspective[35] we investigated commonly used Python plotting libraries and found that Seaborn, Matplotlib, NetworkX, and WordCloud all rely on Matplotlib’s underlying plotting functions. When using these libraries for plotting, a Figure object is generated in memory, which contains all the elements of the plot. This implies that we can extract all visualization-related elements from the Figure object and compare the GT_code with the generated_code to evaluate their visualization effects.

15.2.1 Comparison Between Chartmimic

More Efficient. Unlike ChartMimic [35], which depends on code tracers and code injection, our evaluation method is substantially more efficient. In practice, ChartMimic must execute both the GT_code and generated_code for each evaluation dimension, resulting in up to twelve executions for a single generated_code. This process incurs significant computational overhead in both time and memory. By contrast, our method executes the GT_code and generated_code only once, caches their corresponding Figure objects, and then evaluates multiple dimensions directly on these objects, thereby greatly reducing execution cost.

More General. In comparison to ChartMimic’s [35] hard-coded rules, which exhibit limited adaptability and strong dependence on specific Matplotlib versions, our evaluation method is inherently more general. ChartMimic enforces rule-based matching of plotting elements, which not only imposes strict version constraints but also leaves many elements unsupported. Our approach instead parses the Figure object directly, which comprehensively encapsulates all elements in memory, ensuring greater robustness and version independence.

More Versatile. Whereas ChartMimic [35] is restricted to a narrow set of functions from specific libraries, our method offers broad applicability. By operating directly on core Matplotlib objects, our approach seamlessly extends to all visualization libraries that build upon Matplotlib’s primitives, thereby achieving substantially stronger cross-library generalization.

More Precise. Unlike ChartMimic [35], which evaluates function call patterns rather than visual outputs, our method emphasizes the visualization results themselves. ChartMimic leaves a gap between code execution and rendered charts, while our approach directly inspects visual objects such as Line and Patch. This enables a more faithful and precise evaluation of visualization quality at the code-to-visualization level.

More Dimensions. Our evaluation framework introduces a richer set of dimensions compared to ChartMimic. Specifically, we incorporate grid alignment, legend consistency, visual parameters, and data parameters, enabling a more fine-grained assessment of code-level similarity. By aggregating multiple complementary dimensions, our approach provides a more comprehensive evaluation of similarity from the code perspective. Through analysis of LLM-Score and Base-Score, we observe a high degree of consistency between the two metrics, indicating that the proposed hard-rule matching strategy aligns well with LLM-based assessments to a certain extent. Moreover, Base-Score also reflects visual similarity at the trend level, as illustrated in Fig. 7, Fig. 9 where we analyze the relationship between code-level and chart-level similarity.

15.2.2 Color Score

Traditional approaches typically treat all colors in a chart as an unordered set, neglecting the binding relationship between colors and specific data items[35]. To address this issue, we propose an efficient and more professional method for color extraction strategy designed to parse colors and their corresponding semantic information from Matplotlib’s graphical objects Figure. This strategy decomposes the chart into different types of visual elements and organizes the extracted color information into a structured mapping, which can be expressed as:

{ElementType→{DataKey→HexColor}} (2)

where:

ElementType: Refers to the object to which the color is applied, such as the fill color of a bar chart (patch_face), the line color of a line chart (line_color), the color of a scatter plot (scatter_color), or the background of the axes (axes_bg).

DataKey: Refers to the specific data entity bound to the color. This is typically the label in the legend, the tick label on the axis, or the content of a text element.

HexColor: The standardized hexadecimal color code.

After obtaining the structured color data, we design a set of weighted evaluation metrics to quantify the color fidelity between generated_code and GT_code. The core principle of this evaluation is that not all colors are equally important. For example, errors in the colors of data series are more severe than errors in the colors of axis grid lines.

To this end, we introduce element-type weights (wt), assigning a predefined weight to each ElementType t. Core data elements (e.g., patch_face, line_color) are assigned high weights (e.g., 1.0), whereas auxiliary or decorative elements (e.g., figure_bg, spine) are assigned lower weights (e.g., 0.01).

The evaluation is performed only on the element types and data keys shared by both generated_code(gen) and GT_code(gt). This ensures a valid comparison, avoiding mismatches such as comparing a line color in generated with a bar color in gt_code.

The total weighted similarity Stotal serves as the core of our model, and is computed as:

Stotal=∑t∈T∩∑k∈K∩,twt⋅σ⁡(Cgen,t,k,Cgt,t,k), (3)

where:

Tg​e​n and Tg​t denote the sets of all element types present in the generated chart and the ground-truth chart, respectively.

Kg​e​n,t and Kg​t,t denote the sets of all data keys under element type t in the generated and ground-truth charts, respectively.

wt is the predefined weight for element type t.

Cg​e​n,t,k and Cg​t,t,k are the colors corresponding to element type t and key k in the generated and ground-truth charts, respectively.

σ⁡(C1,C2) is a function measuring the similarity between two hexadecimal colors.

The color similarity function σ⁡(C1,C2) is used to quantify the visual closeness between two colors. In our implementation, we adopt a normalized reversed Euclidean distance in the RgenB color space to compute similarity.

First, the hexadecimal color C is converted into its RGB representation (R,G,B). The Euclidean distance between two colors C1 and C2 is defined as:

d⁡(C1,C2)=(R1−R2)2+(G1−G2)2+(B1−B2)2. (4)

The maximum possible distance in the RGB space corresponds to the distance between (0,0,0) and (255,255,255), i.e.,

dmax=3⋅2552. (5)

We then normalize the distance d and transform it into a similarity score σ within the range [0,1]:

σ⁡(C1,C2)=1−d⁡(C1,C2)dmax. (6)

When two colors are identical, σ=1.0; when they differ maximally, σ=0.0.

To provide comprehensive and interpretable evaluation results, we map the computed total weighted similarity (Stotal) to three standard metrics widely used in the information retrieval domain: Precision, Recall, and F1-Score.

Total Weight: We first compute the total weights of the generated chart and the ground-truth chart, representing the maximum theoretically achievable similarity score.

Wgen=∑t∈Tgen∑k∈Kgen,twt (7)

Wgt=∑t∈Tgt∑k∈Kgt,twt (8)

Precision: Measures the accuracy of all color elements in the generated chart. It answers the question: “Among all generated colors, what proportion is correct?”

Precision=StotalWg​e​n. (9)

Recall: Measures the extent to which all color elements in the ground-truth chart are correctly reproduced in the generated chart. It answers the question: “Among all required colors, what proportion has been correctly generated?”

Recall=StotalWg​t. (10)

F1-Score: The harmonic mean of Precision and Recall, providing a single comprehensive evaluation score.

F1-Score=2⋅Precision⋅RecallPrecision+Recall. (11)

15.2.3 grid Score

We define a structured Grid State Descriptor. For each subplot a​x in a chart, we extract the visibility of its X-axis and Y-axis grid lines, and encode them as a Boolean dictionary:

𝒮grid={’x_grid_visible’:bool’y_grid_visible’:bool (12)

We traverse all Axes objects within a Figure, and for each subplot where at least one grid line (X-axis or Y-axis) is visible, we generate a grid state descriptor. Ultimately, the grid configuration of an entire chart is abstracted as a list of such descriptors, which can be mathematically regarded as a multiset.

For example, in a Figure with two subplots, where the first subplot has only Y-axis grid lines and the second subplot has both X-axis and Y-axis grid lines, the grid configuration is represented as:

1. {’x_grid’:False,’y_grid’:True} (13)

2. {’x_grid’:True,’y_grid’:True}

This structured representation is not only precise but also completely ignores the specific styles of grid lines (e.g., color, linewidth). Instead, it focuses solely on their presence, which captures the core semantics and makes the evaluation more robust.

After extracting the multisets of grid state descriptors from the generated figure (Ggen) and the ground-truth figure (Ggt), we further use the F1 metric to measure the accuracy of this parameter.

We define the following notations:

Ggen: the multiset of grid state descriptors extracted from the generated figure.

Ggt: the multiset of grid state descriptors extracted from the ground-truth figure.

The number of true positives (TP) is defined as the cardinality of the intersection between the two multisets:

T​P=|Ggen∩Ggt|. (14)

True Positives (TP) A true positive is defined as a grid state descriptor that appears in Gg​e​n and exactly matches one in Gg​t. The total number of true positives is given by the size of the intersection of these two multisets:

T​P=|Gg​e​n∩Gg​t|. (15)

Precision Precision measures the proportion of correctly activated grid configurations among all grid configurations in the generated figure (i.e., those that also exist in the ground-truth figure):

Precision=T​P|Gg​e​n|=|Gg​e​n∩Gg​t||Gg​e​n|. (16)

If |Gg​e​n|=0, we define Precision=1.0.

Recall Recall measures the proportion of required grid configurations in the ground-truth figure that are successfully reproduced in the generated figure:

Recall=T​P|Gg​t|=|Gg​e​n∩Gg​t||Gg​t|. (17)

If |Gg​t|=0, we define Recall=1.0.

F1-Score The F1-score, as the harmonic mean of precision and recall, provides a single comprehensive metric:

F1-Score=2⋅Precision⋅RecallPrecision+Recall. (18)

15.2.4 Layout score

For each individual subplot (i.e., an Axes object) in a chart, we create a unique and quantitative Layout Descriptor. This descriptor fully defines the size and position of the subplot within a virtual grid (GridSpec). Instead of relying on pixel coordinates, we extract the underlying structural information from Matplotlib’s SubplotSpec object.

For each subplot a​x in a Figure, we extract the following six key parameters to construct its layout descriptor D:

n​r​o​w​s (R): the total number of rows in the corresponding GridSpec.

n​c​o​l​s (C): the total number of columns in the corresponding GridSpec.

r​o​w​_​s​t​a​r​t (rs): the starting row index of the grid cells occupied by the subplot.

r​o​w​_​e​n​d (re): the ending row index of the grid cells occupied by the subplot.

c​o​l​_​s​t​a​r​t (cs): the starting column index of the grid cells occupied by the subplot.

c​o​l​_​e​n​d (ce): the ending column index of the grid cells occupied by the subplot.

Thus, the layout of each subplot can be precisely represented as a 6-tuple:

D=(R,C,rs,re,cs,ce). (19)

By traversing all Axes objects in a Figure, the overall layout can be abstracted as a multiset of these layout descriptors D, denoted as L.

We define the following notation:

Lg​e​n: the multiset of layout descriptors extracted from the generated figure.

LG​T: the multiset of layout descriptors extracted from the ground-truth figure.

True Positives (TP) A true positive represents a layout descriptor that exists in Lg​e​n and exactly matches one in Lg​t. The total number of true positives is defined as the size of the intersection of these two multisets:

T​P=|Lg​e​n∩Lg​t| (20)

This indicates the number of subplots that are correctly generated and placed in the correct positions.

Precision Precision measures the proportion of correctly generated subplots among all generated subplots:

Precision=T​P|Lg​e​n|=|Lg​e​n∩Lg​t||Lg​e​n| (21)

Here, |Lg​e​n| denotes the total number of subplots in the generated figure. A low precision indicates that the model produced redundant or incorrectly placed subplots.

Recall Recall measures the proportion of required subplots in the ground-truth figure that were successfully generated:

Recall=T​P|Lg​t|=|Lg​e​n∩Lg​t||Lg​t| (22)

Here, |Lg​t| denotes the total number of subplots in the ground-truth figure. A low recall suggests that the model failed to generate all required subplots.

F1-Score The F1-score, as the harmonic mean of precision and recall, provides a single balanced metric for evaluating the overall quality of the layout:

F1-Score=2⋅Precision⋅RecallPrecision+Recall (23)

15.2.5 Legend score

We propose a Dual-Constraint Matching Framework for Legend Evaluation. This framework decomposes legend evaluation into independent assessments of the semantic and spatial properties of each individual legend entry, and quantifies the consistency between the generated and ground-truth figures through a flexible matching algorithm. Consequently, it provides a more comprehensive and robust evaluation scheme.

Our method does not treat the legend as a single entity but decomposes it into a collection of independent legend entries. For each visible legend object in the chart, we traverse all its text labels and create an atomic, structured Legend Descriptor for each label.

The descriptor D is defined as a 2-tuple that captures both semantic and spatial information:

D=(t,B) (24)

where:

t is a string representing the textual content of the legend entry. This element captures the semantic correctness of the legend.

B is a 4-tuple (x0,y0,x1,y1) representing the bounding box of the entire legend object containing the text entry, expressed in the screen rendering coordinate system. This element captures the spatial correctness of the legend.

By traversing all legends from both the Axes objects and the Figure object itself, we can extract all visible legend entries of a chart and represent them collectively as a multiset of descriptors D, denoted as L.

After extracting the multisets of legend descriptors Lg​e​n and Lg​t from the generated and ground-truth figures, respectively, we design a dual-constraint matching algorithm to compute their similarity. The algorithm can flexibly operate in two modes: semantic-only matching or combined semantic and spatial matching.

A descriptor Dg​e​n=(tg​e​n,Bg​e​n) from Lg​e​n matches a descriptor Dg​t=(tg​t,Bg​t) from Lg​t if and only if one or both of the following constraints are satisfied:

Semantic Constraint: The text content of the two descriptors must be identical:

tg​e​n=tg​t. (25)

Positional Constraint: The bounding boxes of the legend objects containing the descriptors must have a positive intersection area:

Areai​n​t​e​r​s​e​c​t​i​o​n​(Bg​e​n,Bg​t)>0. (26)

For two bounding boxes B1=(x1,0,y1,0,x1,1,y1,1) and B2=(x2,0,y2,0,x2,1,y2,1), the intersection area is computed as:

xA =max⁡(x1,0,x2,0),yA=max⁡(y1,0,y2,0) (27)

xB =min⁡(x1,1,x2,1),yB=min⁡(y1,1,y2,1)

Areaint=max⁡(0,xB−xA)

⋅max⁡(0,yB−yA)

The algorithm finds unique matching pairs that satisfy the above constraints (removing matched descriptors from the pool) and computes the total number of true positives (TP). Based on TP, we perform the final quantitative evaluation using standard precision, recall, and F1-score metrics:

Precision=T​P|Lg​e​n|,Recall=T​P|Lg​t|,F1-Score=2⋅Precision⋅RecallPrecision+Recall. (28)

15.2.6 data parameter score

The primary goal of data visualization is to faithfully and accurately convey the underlying data. We introduce an evaluation framework designed to quantify the fidelity of a chart’s data parameters. This framework inspects the chart at a deep level, directly verifying the correctness of its underlying data.

The first step of the framework is to identify and extract the data parameters that directly define the data representation of the chart. Through introspection of Matplotlib plotting elements, we categorize these parameters into distinct types. The set of data parameters, denoted as Kd​a​t​a, is explicitly defined as:

Kdata={’xdata’,’ydata’,’offsets’,’xy’,’verts’,’width’,’height’,’sizes’}. (29)

These parameters directly correspond to the geometric and positional properties of chart elements:

For line plots (Line2D), we extract xdata and ydata.

For bar charts (Rectangle), we extract the lower-left corner coordinates xy, as well as width and height.

For filled plots (Polygon), we extract all vertex coordinates verts.

For scatter plots (Collection), we extract the center coordinates offsets and the point sizes sizes.

Through this process, each chart is decomposed into a multiset E of element-parameter dictionaries.

Data parameters, especially those represented as arrays, cannot be compared using simple equality operators. To robustly handle variations in data point ordering or floating-point precision, we define a dedicated similarity function S⁡(v1,v2). The core logic for data parameters is as follows:

Numeric Type: For scalar values, we use numpy.isclose to determine whether two floating-point numbers are approximately equal within a tolerance ϵ:

S⁡(v1,v2)={1if ​|v1−v2|≤ϵ0otherwise (30)

Array-like Type: For array data, which is crucial for evaluating data parameters, we adopt the Jaccard similarity coefficient to measure the overlap between the contents of two arrays. Let V1 and V2 denote the sets of elements in v1 and v2, respectively:

S⁡(v1,v2)=|V1∩V2||V1∪V2| (31)

This method is insensitive to the order of data points and accurately reflects the true content overlap between two datasets.

After quantifying the similarity between parameters, we employ a two-stage algorithm to compute the final evaluation metrics.

Element Matching: To address differences in element order and quantity across charts, we use a greedy optimal matching algorithm. For each element eg​t in the ground-truth chart, the algorithm searches among elements of the same type in the generated chart to find the best match eg​e​n∗ that maximizes the total similarity across all parameters. This matching is performed globally, considering all parameter types. The result is a set of successful matches:

M={(eg​e​n,eg​t)}. (32)

Data Metric Computation: Once the matching set M is obtained, we focus exclusively on data parameters to aggregate the scores. The total true positive score for the data dimension, T​Pd​a​t​a, is computed as the sum of similarities across all matched pairs. We iterate over the union of keys to ensure penalties for missing or extra parameters:

TP=∑(egen,egt)∈M∑k∈𝒦eS⁡(egen​[k],egt​[k]) (33)

𝒦e=(keys​(egen)∪keys​(egt))∩Kdata, Next, we count the total number of data parameters in the generated chart and the ground-truth chart, denoted as Nd​a​t​a,g​e​n and Nd​a​t​a,g​t, respectively. Finally, we compute the precision, recall, and F1-score for the data dimension:

Precision =T​PNd​a​t​a,g​e​n, (34)

Recall =T​PNd​a​t​a,g​t,

F1-Score =2⋅Precision⋅RecallPrecision+Recall.

15.2.7 visual parameter score

The visual style of a chart is also an important component of chart reproduction quality. Visual style is governed by a set of visual parameters, such as line styles, marker shapes, element transparency, and so on. Correct usage of these parameters not only affects the aesthetic quality and professionalism of the chart, but also directly determines whether it adheres to specific design guidelines or user instructions. We propose a framework, running in parallel with the data parameter evaluation, specifically designed to quantify the consistency of a chart with respect to its visual parameters.

This framework builds upon the parameterized representation established in 15.2.6. After extracting all parameters of an element, we identify the set of visual parameters (Kv​i​s​u​a​l) by exclusion. A parameter key k is classified as a visual parameter if it satisfies:

k∉Kd​a​t​aandk∉Ki​g​n​o​r​e (35)

where Kd​a​t​a is the predefined set of data parameters, and Ki​g​n​o​r​e is the set of parameters handled by other evaluators (e.g., color). Typical visual parameters include: ’linestyle’, ’linewidth’, ’marker’, ’markersize’, ’alpha’, and so on. The extraction process is performed in parallel with that of the data parameters, but subsequent evaluation computations focus exclusively on this subset of parameters.

We employ the same general similarity function S⁡(v1,v2) introduced in the (30) and (31) to compare the values of visual parameters. Its robustness is equally applicable to various data types of visual parameters:

String type: For parameters such as linestyle (e.g., ’-’ vs ’–’) or marker (e.g., ’o’ vs ’x’), the function performs a direct string equality comparison.

Numeric type: For parameters such as linewidth (e.g., 1.5 vs 2.0) or alpha (e.g., 0.8 vs 1.0), the function uses numpy.isclose to perform a tolerance-based comparison.

This consistent definition of similarity ensures intrinsic coherence across different evaluation dimensions.

Element Matching: We reuse the set of matched element pairs M={(eg​e​n,eg​t)} obtained through the greedy optimal matching algorithm. This implies that the matching of elements is determined based on their overall similarity (data + visual), consistent with human perception — we always perceive an element as a whole. Establishing a match indicates that both the data and visual aspects will be evaluated for that pair.

Visual Metric Computation: Given the set of matched pairs M, we focus exclusively on the visual parameters to aggregate the scores. We compute the total true positive score for the visual dimension (T​Pv​i​s​u​a​l), defined as the sum of visual parameter similarities across all matched pairs:

TPvisual=∑(egen,egt)∈M∑k∈𝒦vS⁡(egen​[k],egt​[k]) (36)

Similarly, we count the total number of visual parameters in the generated and ground-truth charts, denoted as Nv​i​s​u​a​l,g​e​n and Nv​i​s​u​a​l,g​t, respectively. Finally, the precision, recall, and F1-score for the visual dimension are computed as:

Precisionv​i​s​u​a​l =T​Pv​i​s​u​a​lNv​i​s​u​a​l,g​e​n, (37)

Recallv​i​s​u​a​l =T​Pv​i​s​u​a​lNv​i​s​u​a​l,g​t,

F1-Scorev​i​s​u​a​l =2⋅Precisionv​i​s​u​a​l⋅Recallv​i​s​u​a​lPrecisionv​i​s​u​a​l+Recallv​i​s​u​a​l.

15.2.8 type score

We propose an evaluation framework based on Artist Class Introspection. Unlike methods that rely on the visual rendering of charts, this framework directly inspects the object model constructed in memory by the plotting library (Matplotlib). By examining the core drawing artists (i.e., primitive graphical objects) and their associated classes, the framework deterministically and robustly infers the composition of a chart. The key idea is that Matplotlib employs different classes of artist objects for different types of plots. For example, a line plot is rendered using Line2D objects, whereas a bar chart is rendered using Rectangle objects. Leveraging this intrinsic correspondence, we can infer the chart types present in a figure by identifying which classes of artist objects it contains.

Our algorithm operates by traversing all subplots (Axes) within a matplotlib.Figure object and inspecting the list of artists contained in each subplot (e.g., ax.lines, ax.patches, ax.collections, etc.).

The algorithm aggregates all detected chart types within a figure into a set. This set-based representation has a significant advantage: it naturally supports the identification and evaluation of composite charts. For example, a chart that overlays a line plot on top of a bar chart will be recognized as containing both bar_or_hist and line.

The number of true positives is defined as the size of the intersection between the two sets, that is, the number of chart types present in both the generated chart and the reference chart:

T​P=|Tgen∩Tgt| (38)

Precision measures the proportion of correct chart types among all generated chart types:

Precision=T​P|Tgen|=|Tgen∩Tgt||Tgen| (39)

where |Tgen| denotes the total number of distinct chart types detected in the generated chart.

Recall measures the proportion of reference chart types that are successfully generated:

Recall=T​P|Tgt|=|Tgen∩Tgt||Tgt| (40)

where |Tgt| denotes the total number of distinct chart types in the reference chart.

The F1-Score is the harmonic mean of precision and recall, providing a comprehensive evaluation metric:

F1-Score=2⋅Precision⋅RecallPrecision+Recall (41)

15.2.9 text score

We propose a text evaluation framework based on semantic categorization and fuzzy matching. In this framework, all textual elements in a chart are categorized according to their functional roles, and a fuzzy matching algorithm based on edit distance is applied among texts within the same category. This enables a quantitative evaluation of chart text that is both strict and robust.

To achieve precise evaluation of textual roles, we first design an extractor (_extract_texts_from_figure) that introspects the matplotlib Figure object to identify and classify all visible textual elements. Instead of treating all texts as an undifferentiated set, we categorize them into predefined semantic classes.

Through this process, the entire textual content of a chart is transformed into a structured Text Map, denoted as T. Its form is a dictionary that maps each category name to the list of text strings belonging to that category: T={c→[t1,t2,…]}. For example, Ttitle represents the list of all subplot titles in the figure. This categorization mechanism ensures context-aware evaluation and prevents, for instance, an axis label from being incorrectly compared with a title.

After obtaining the text maps of the generated chart and the reference chart, Tgen and Tgt, we designed an evaluation algorithm to quantify their consistency. To tolerate minor textual differences, we adopt the Levenshtein Ratio as the similarity function between two strings s1 and s2, denoted as SL​(s1,s2). This function is based on computing the minimum number of single-character edits (insertions, deletions, or substitutions) required to transform one string into the other (i.e., the Levenshtein Distance), and normalizes the value to the interval [0,1]:

SL​(s1,s2)=1−LevenshteinDistance​(s1,s2)max⁡(|s1|,|s2|) (42)

A higher value of SL indicates greater similarity between the two strings. Identical strings achieve a similarity of 1.

Our evaluation algorithm operates independently within each semantic category. For each category c, the algorithm searches for the best match tg​t∗ for every generated text tgen∈Tgen,c from the available reference texts Tgt,c, such that SL​(tgen,tg​t) is maximized. To prevent one-to-many matches, once a reference text is matched, it is removed from the candidate pool.

We then accumulate the similarity scores of all best matches across all categories to obtain a total similarity score (T​Pscore), which can be regarded as a weighted sum of “true positives”:

T​Pscore=∑c∈C∑tgen∈Tgen,cmaxtg​t∈Tgt,c′⁡SL​(tgen,tg​t) (43)

where C denotes the union of all text categories present in both charts, and Tgt,c′ is the set of unmatched reference texts in category c.

Finally, we compute the total number of generated and reference texts (Ngen and Ngt), and derive the Precision, Recall, and F1-Score as follows:

Precision=T​PscoreNgen,Ngen=∑c|Tgen,c| (44)

Recall=T​PscoreNgt,Ngt=∑c|Tgt,c| (45)

F1-Score=2⋅Precision⋅RecallPrecision+Recall (46)

15.3 LLM-Evaluation

This study designs and implements a multi-dimensional visualization code evaluation framework based on Large Language Models (LLMs). The framework does not execute code or render images; instead, it leverages the powerful code understanding and reasoning capabilities of LLMs to perform static analysis directly on the source code of both the generated and reference scripts. By decomposing the complex problem of “visual similarity” into a series of well-defined and mutually orthogonal evaluation dimensions, and by designing strict scoring instructions for each, our framework provides a comprehensive, in-depth, and interpretable quantitative assessment of chart code quality.

We deconstruct the ambiguous task of “code quality” assessment into eight specific and independent evaluation dimensions, denoted as Di. This approach makes the LLM’s evaluation task more focused and renders the final results more diagnostic and interpretable. The eight dimensions are defined as follows:

Data Logic (Dd​a​t​a): Evaluates the integrity of data presentation, including raw values, sorting, filtering, aggregation, and mathematical transformations, ensuring the quantitative information conveyed is identical to the ground truth.

Chart Type and Geometry (Dt​y​p​e): Evaluates the correctness of the core plotting primitive (e.g., scatter, bar, contour) and the dimensionality of the visualization.

Color and Color Mapping (Dc​o​l​o​r): Evaluates the final rendered colors, colormap families, direction, and normalization ranges, distinguishing between data-driven mappings and fixed style choices.

Visual Parameters (Dv​i​s): Evaluates non-color stylistic attributes such as line width, marker styles, transparency (alpha), and line styles.

Layout and Structure (Dl​a​y​o​u​t): Evaluates the overall figure configuration, including canvas size, aspect ratio, subplot grid structures, and spacing adjustments (e.g., tight layout).

Legend Configuration (Dl​e​g​e​n​d): Evaluates the presence, content, ordering, and positioning of the legend, ensuring it accurately reflects the data series.

Grid and Axes (Dg​r​i​d): Evaluates the configuration of coordinate systems, including grid line visibility/style, axis limits, spines, and tick specifications.

Text Content (Dt​e​x​t): Evaluates the accuracy of all textual annotations, including titles, axis labels, and specific data point annotations.

15.4 LMM-Evaluation

The ultimate criterion for evaluating automatically generated charts should be human visual perception. Although programmatic evaluation and source code analysis can technically ensure the correctness of chart components and parameters, they may not fully capture all visual details, artifacts, or the overall aesthetic coherence in the final rendered image. To establish an evaluation system that more closely approximates a "gold standard," we argue for the necessity of directly assessing the final visual output—the chart image itself.

To this end, this study designs and implements a holistic chart image evaluation framework based on Vision-Language Models (VLMs). This framework utilizes advanced multimodal large models by simultaneously providing them with both the reference and the generated images, supplemented by a set of rigorous evaluation instructions, to directly quantify the visual similarity between the two. This end-to-end visual evaluation method can capture a wide range of discrepancies, from macroscopic layout to microscopic pixel-level differences, thereby providing a comprehensive and holistic quality score. Here, we adopt a holistic evaluation approach, assessing all visual aspects in a single call. To ensure rigor, we extend and reinforce the philosophy of a deduction-based scoring system. The instructions require the model to assume a perfect score of 100, and then to deduct points for every visual discrepancy it finds between the two images.

The evaluation prompt is in 22.7

Table 19: Run configurations for all models. Unset values indicate that their default values are being used. For Proprietary models, we are unable to use a Top-P of exactly 1 due to their API settings, and we end up using a value of 0.99999. Temp. denotes temperature. We use model pages’ code to set up the run configurations whenever possible.

Model Version/HF Checkpoint Do Sample level 1 2 Max level 3 Max Temp. Top-P

Proprietary Multimodal Large Language Models

GPT-5.2 [22] gpt-5.2-2025-12-11 default 55000 0.1 1

Claude 4 Sonnet [1] claude-4-sonnet-20250523 default 55000 0.1 1

Gemini-3-Pro [5] gemini-3-pro-20251118 default 55000 0.1 1

doubao-seed-1-5 [9] seed1.5-VL-20250513 default 16000 0.1 1

doubao-seed-1-6 [26] seed1.6-VL-20250625 default 32768 0.1 1

Open-Source Multimodal Large Language Models

Qwen2-VL-7B [27] Qwen/Qwen2-VL-7B-Instruct True 8192 32768 0.1 0.95

Qwen2-VL-72B [27] Qwen/Qwen2-VL-72B-Instruct True 8192 32768 0.1 0.95

Qwen2.5-VL-7B [3] Qwen/Qwen2.5-VL-7B-Instruct True 8192 32768 0.1 0.95

qwen2.5-VL-72B [3] Qwen/Qwen2.5-VL-72B-Instruct True 8192 32768 0.1 0.95

deepseek-VL-7B [18] deepseek-ai/deepseek-vl-7b-base True 8192 32768 0.1 0.95

kimi-VL-A3B [25] moonshotai/Kimi-VL-A3B-Thinking True 8192 32768 0.1 0.95

MiMo-VL-7B-RL [31] XiaomiMiMo/MiMo-VL-7B-RL-2508 True 8192 32768 0.1 0.95

MiMo-VL-7B-SFT [31] XiaomiMiMo/MiMo-VL-7B-SFT-2508 True 8192 32768 0.1 0.95

GLM-4-9b [7] zai-org/glm-4-9b True 8192 32768 0.1 0.95

GLM-4.6V-Flash [41] zai-org/glm-4.6v-Flash True 8192 32768 0.1 0.95

Intern-VL 2.5 8B [4] OpenGVLab/InternVL2_5-8B True 8192 32768 0.1 0.95

Intern-VL 2.5 38B [4] OpenGVLab/InternVL2_5-38B True 8192 32768 0.1 0.95

Intern-VL 3 8B [42] OpenGVLab/InternVL3-8B True 8192 32768 0.1 0.95

Intern-VL 3 38B [42] OpenGVLab/InternVL3-38B True 8192 32768 0.1 0.95

Intern-VL 3.5 8B [28] OpenGVLab/InternVL3_5-8B True 8192 32768 0.1 0.95

Intern-VL 3.5 38B [28] OpenGVLab/InternVL3_5-38B True 8192 32768 0.1 0.95

llava-onevision-qwen2-7b-si [12] lmms-lab/llava-onevision-qwen2-7b-si True 8192 32768 0.1 0.95

llava-onevision-qwen2-7b-ov [12] lmms-lab/llava-onevision-qwen2-7b-ov True 8192 32768 0.1 0.95

Qwen3-VL-30B-A3B [2] Qwen/Qwen3-VL-30B-A3B-Instruct True 8192 32768 0.1 0.95

Qwen3-VL-30B-A3B 3.5 38B [2] Qwen/Qwen3-VL-30B-A3B-Think True 8192 32768 0.1 0.95

Qwen3-VL-32B [2] Qwen/Qwen3-VL-32B-Instruct True 8192 32768 0.1 0.95

16 Run configurations

We summarize the run configurations for all evaluated models in Table 19. To ensure a fair and consistent comparison across different systems, we adopt a unified configuration strategy whenever possible.

For proprietary models, we primarily rely on their official APIs and follow the default settings provided by the platforms, with minimal modifications. In particular, due to API constraints, we set the Top-P value to 0.99999 instead of exactly 1. The temperature is fixed at 0.1 to reduce randomness and improve output stability.

For open-source models, we enable stochastic decoding (do_sample=True) and use a consistent configuration with a context length of 8192 tokens for Level 1 and Level 2 tasks, and up to 32768 tokens for Level 3 tasks. These settings are chosen to balance generation quality and computational efficiency.

Whenever possible, we follow the official implementation or model documentation to configure inference parameters. This ensures that each model is evaluated under conditions that are both reproducible and representative of its intended usage.

17 Open-Source Model Components

We summarize the key components of all evaluated open-source multimodal models in Table 20, including their vision encoders, language backbones, and input resolutions.

The evaluated models exhibit diverse architectural designs. Most models adopt variants of Vision Transformers (ViT) or CLIP-based encoders (e.g., Qwen-VL, InternVL, and GLM-4V), while others incorporate hybrid designs such as SigLIP combined with SAM for enhanced visual representation (e.g., DeepSeek-VL). On the language side, models are typically built upon large language model backbones such as Qwen, InternLM, or custom proprietary architectures.

Regarding input resolution, we follow the default settings of each model whenever possible, denoted as original in the table. This is particularly important for models that support dynamic resolution inputs. For models with stricter input requirements (e.g., DeepSeek-VL-7B and GLM-4-9B), we apply a maximum resolution constraint to ensure compatibility with their architectures.

This diversity in model components reflects the current design space of multimodal systems and provides a comprehensive basis for evaluating their performance under realistic settings.

Table 20: We summarize the visual and language components of the open-source models evaluated in our benchmark, along with the input resolutions used in our evaluation. Here, original denotes that we use the default image size, as the corresponding models support dynamic resolution inputs. Note that for DeepSeekVL-7B and GLM-4-9B , we apply a maximum input size constraint to accommodate their requirements.

Model Vision Language Resolu-

Encoder Model tion

Qwen2-VL-7B Qwen2-VL ViT-14-224 Qwen2-VL-LLM-7B origianl

Qwen2-VL-72B Qwen2-VL ViT-14-224 Qwen2-VL-LLM-72B origianl

Qwen2.5-VL-7B Qwen2.5-VL ViT-14-224 Qwen2.5-VL-LLM-7B origianl

Qwen2.5-VL-72B Qwen2.5-VL ViT-14-224 Qwen2.5-VL-LLM-72B origianl

Deepseek-VL-7B SigLIP-384-SO400M & DeepSeek-LLM-7B 1152×1152*

SAM-ViT-Base

Kimi-VL-A3B MoonViT Moonlight Model origianl

MiMo-VL Qwen2.5-ViT MiMo-7B origianl

GLM-4-9B CLIP ViT-L-14-336 InternLM-7B 1120×1120*

InternVL-2.5-8B InternViT-6B-448px-V2_5 internlm2_5-7b-chat origianl

InternVL-2.5-38B InternViT-6B-448px-V2_5 Qwen2.5-32B-Instruct origianl

InternVL-3-8B InternViT-300M-448px-V2_5 Qwen2.5-7B origianl

InternVL-3-38B InternViT-6B-448px-V2_5 Qwen2.5-32B origianl

InternVL-3.5-8B InternViT-300M & Qwen3-8B origianl

InternViT-6B

InternVL-3.5-38B InternViT-300M & Qwen3-38B origianl

InternViT-6B

llava-onevision-qwen2-7b-si SigLIP-384-SO400M Qwen2-7B origianl

llava-onevision-qwen2-7b-ov SigLIP-384-SO400M Qwen2-7B origianl

Table 21: The release time and model source of LMMs used in our benchmark.

Model Release Time Source

Closed-source Models

GPT-5.2 2025-12-11 https://openai.com/zh-Hans-CN/index/introducing-gpt-5-2/

Claude 4 Sonnet 2025-05-23 https://www.anthropic.com/news/claude-4

Gemini-3-Pro 2025-11-18 https://deepmind.google/models/gemini/pro/

doubao-seed-1.5 2025-05-11 https://www.volcengine.com/product/doubao

doubao-seed-1.6 2025-06-11 https://www.volcengine.com/product/doubao

Open-source Models

Qwen2-VL-7B 2024-09-18 https://huggingface.co/Qwen/Qwen2-VL-7B-Instruct

Qwen2-VL-72B 2024-09-18 https://huggingface.co/Qwen/Qwen2-VL-72B-Instruct

qwen2.5-VL-7B 2025-01-26 https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct

qwen2.5-VL-72B 2025-01-26 https://huggingface.co/Qwen/Qwen2.5-VL-72B-Instruct

deepseek-VL-7B 2024-03-09 https://huggingface.co/deepseek-ai/deepseek-vl-7b-base

kimi-VL-A3B 2024-08-20 https://huggingface.co/moonshotai/Kimi-VL-A3B-Thinking

MiMo-VL-7B-RL 2025-08-10 https://huggingface.co/XiaomiMiMo/MiMo-VL-7B-RL-2508

MiMo-VL-7B-SFT 2025-08-10 https://huggingface.co/XiaomiMiMo/MiMo-VL-7B-SFT-2508

GLM-4-9B 2024-06-19 https://huggingface.co/zai-org/glm-4-9b

Intern-VL 2.5 8B 2024-11-21 https://huggingface.co/OpenGVLab/InternVL2_5-8B

Intern-VL 2.5 38B 2024-11-21 https://huggingface.co/OpenGVLab/InternVL2_5-38B

Intern-VL 3 8B 2025-04-10 https://huggingface.co/OpenGVLab/InternVL3-8B

Intern-VL 3 38B 2025-04-10 https://huggingface.co/OpenGVLab/InternVL3-38B

Intern-VL 3.5 8B 2025-08-25 https://huggingface.co/OpenGVLab/InternVL3_5-8B

Intern-VL 3.5 38B 2024-08-25 https://huggingface.co/OpenGVLab/InternVL3_5-38B

llava-onevision-qwen2-7b-si 2024-07-29 https://huggingface.co/lmms-lab/llava-onevision-qwen2-7b-si

llava-onevision-qwen2-7b-ov 2024-07-25 https://huggingface.co/lmms-lab/llava-onevision-qwen2-7b-ov

Qwen3-VL-30B-A3B-Instruct 2025-10-03 https://huggingface.co/Qwen/Qwen3-VL-30B-A3B-Instruct

Qwen3-VL-30B-A3B-Thinking 2025-10-03 https://huggingface.co/Qwen/Qwen3-VL-30B-A3B-Thinking

Qwen3-VL-32B-Instruction 2025-10-23 https://huggingface.co/Qwen/Qwen3-VL-32B-Instruct

GLM-4.6v-Flash 2025-12-08 https://huggingface.co/zai-org/GLM-4.6V-Flash

18 Model License Information

To ensure transparency and clarify the usage conditions of all models involved in our experiments, we provide a summary of their licensing information in Table 22. This table reports both the model license and the code license for each evaluated model.

The model license refers to the terms governing the distribution and usage of the pretrained model weights or API services, while the code license corresponds to the licensing of the associated implementation, including training or inference code when available. For proprietary models, both licenses are typically governed by platform-specific terms of service rather than open-source licenses.

We include a diverse set of models, ranging from proprietary multimodal large language models (e.g., GPT-5.2, Claude 4 Sonnet, Gemini-3-Pro) to open-source models released under permissive licenses such as Apache 2.0 and MIT. This diversity reflects the practical landscape of current multimodal systems and enables a comprehensive evaluation across different licensing regimes.

Entries marked with “Not Applicable” indicate that no explicit license information is publicly available in the corresponding repository or model documentation. In such cases, users should refer to the official platform or provider for the most up-to-date usage terms.

Table 22: Summary of licenses in models that are evaluated in Chart2Code. Entries marked with “Not Applicable” indicate that authors do not have an explicit code license displayed within the codebase or model checkpoint page.

Name Model License Code License

GPT-5.2 Proprietary Proprietary

Claude 4 Sonnet Proprietary Proprietary

Gemini-3-Pro Proprietary Proprietary

doubao-seed-1.6 Proprietary Proprietary

doubao-seed-1.5 Proprietary Proprietary

Qwen2-VL-7B qwen Apache 2.0

Qwen2-VL-72B qwen Apache 2.0

qwen2.5-VL-7B qwen Apache 2.0

qwen2.5-VL-72B qwen Apache 2.0

qwen3-VL-30B-A3B qwen Apache 2.0

qwen3-VL-30B-Instruct qwen Apache 2.0

deepseek-VL-7B deepseek MIT

kimi-VL-A3B MIT MIT

MiMo-VL-7B-RL MIT Apache 2.0

MiMo-VL-7B-SFT MIT Apache 2.0

GLM-4-9B glm-4 Apache 2.0

GLM-4.6v-Flash glm-4 Apache 2.0

Intern-VL 2.5 8B Apache-2.0 MIT

Intern-VL 2.5 38B Apache-2.0 MIT

Intern-VL 3 8B Apache-2.0 MIT

Intern-VL 3 38B Apache-2.0 MIT

Intern-VL 3.5 8B Apache-2.0 MIT

Intern-VL 3.5 38B Apache 2.0 MIT

llava-onevision-qwen2-7b-si Apache 2.0 Apache 2.0

llava-onevision-qwen2-7b-ov Apache 2.0 Apache 2.0

19 Model Source

We summarize the release dates and official sources of all evaluated models in Table 21. This information provides transparency regarding the provenance of each model and ensures the reproducibility of our benchmark.

Our evaluation includes both proprietary and open-source models. For proprietary systems, we reference their official product pages or announcements (e.g., OpenAI, Anthropic, and Google DeepMind). For open-source models, we provide links to their corresponding repositories or checkpoints, primarily hosted on platforms such as Hugging Face.

The release timeline spans from early 2024 to late 2025, covering multiple generations of multimodal models. This temporal diversity allows us to evaluate both earlier and more recent architectures, providing a more comprehensive view of the current landscape.

By explicitly documenting model sources and release dates, we aim to ensure that all results in our benchmark are traceable and verifiable.

20 Sample of Data

To provide a clearer understanding of the structure and characteristics of our benchmark, we present representative examples from our dataset in this appendix.

Instruction: You are a Python developer proficient in data visualization, with expertise in using libraries such as Matplotlib, NetworkX, Seaborn, and others.I have a plot generated by Python code, but I don’t have the corresponding code that generated this plot. Your task is to generate the Python code that can perfectly reproduce the picture based on the image I provide.

Here are the requirements for the task: 1. Data Extraction: Extract the actual data from the provided image. Based on the visual features of the plot, you must infer the data and recreate the plot. 2. Recreate the Image: Generate the Matplotlib code that reproduces the image exactly as it appears, including all elements such as: - Plot type (scatter, line, bar, etc.) - Axis labels and titles - Colors, markers, line styles, and other visual styles - Any legends, annotations, or gridlines present in the image 3. Self-contained Code: The Python code should be complete, executable, and self-contained. It should not require any external data files or variables not already present in the code. Your objective is to extract the any necessary details from the image and generate a Python script that accurately reproduces the plot.

Now, please generate the Python code to reproduce the picture below.

Reference figure:

GT Code:

Instruction: You are a Python developer proficient in data visualization, with expertise in using libraries such as Matplotlib, NetworkX, Seaborn, and others.I have a plot generated by Python code, but I don’t have the corresponding code that generated this plot. Your task is to generate the Python code that can perfectly reproduce the picture based on the image I provide.

Here are the requirements for the task: 1. Data Extraction: Extract the actual data from the provided image. Based on the visual features of the plot, you must infer the data and recreate the plot. 2. Recreate the Image: Generate the Matplotlib code that reproduces the image exactly as it appears, including all elements such as: - Plot type (scatter, line, bar, etc.) - Axis labels and titles - Colors, markers, line styles, and other visual styles - Any legends, annotations, or gridlines present in the image 3. Self-contained Code: The Python code should be complete, executable, and self-contained. It should not require any external data files or variables not already present in the code. Your objective is to extract the any necessary details from the image and generate a Python script that accurately reproduces the plot.

Now, please generate the Python code to reproduce the picture below.

Reference figure:

GT Code:

Instruction: You are a Python developer proficient in data visualization, with expertise in using libraries such as Matplotlib, NetworkX, Seaborn, and others.I have a plot generated by Python code, but I don’t have the corresponding code that generated this plot. Your task is to generate the Python code that can perfectly reproduce the picture based on the image I provide.

Here are the requirements for the task: 1. Data Extraction: Extract the actual data from the provided image. Based on the visual features of the plot, you must infer the data and recreate the plot. 2. Recreate the Image: Generate the Matplotlib code that reproduces the image exactly as it appears, including all elements such as: - Plot type (scatter, line, bar, etc.) - Axis labels and titles - Colors, markers, line styles, and other visual styles - Any legends, annotations, or gridlines present in the image 3. Self-contained Code: The Python code should be complete, executable, and self-contained. It should not require any external data files or variables not already present in the code. Your objective is to extract the any necessary details from the image and generate a Python script that accurately reproduces the plot.

Now, please generate the Python code to reproduce the picture below.

Reference figure:

GT Code:

Instruction: I want to use a heatmap to show the variation range of each category for each month, with the horizontal axis representing time and the vertical axis representing the three categories: Energy, Metals, and Food. The color intensity represents the magnitude of the variation. Please refer to the uploaded image style to generate runnable Python code.

Reference figure:

Raw data: "dates": [ "2020-01-01", "2020-02-01", … "2024-08-01", "2024-09-01" ], "commodities": [ "Energy", "Metals", "Food" ], "values": [ [ 4.7, … -8.1 ], [ 1.6, … -4.7 ], [ 8.8, … -0.3 ] ]

GT Code:

GT Figure:

Instruction: I want to use the data from the uploaded director compensation table (PNG) and create a combination chart based on the style of the reference combination chart: the horizontal axis represents the names of the directors, the bar chart displays cash compensation, stock awards, and total compensation respectively, and a dashed line chart highlights the trends of these three items. Thank you! Adjust the image size to match the aspect ratio of the reference image; use the dark blue, cyan, and light gray tones from the reference image; for the x-axis labels, tilt them 45 degrees and align them to the right, mimicking the text style of the reference image; add a title centered at the top, with font effects similar to the reference image; set the y-axis scale range and intervals according to the reference image; keep the legend position consistent with the reference image, arranged horizontally at the top; apply dashed line styles as in the reference image, and mimic the marker shapes from the reference image.

Reference figure:

Data figure:

GT Code:

GT Figure:

Instruction: Use GridSpec to create a complex 1+2 layout. The top section will feature a large subplot (spanning the entire width) to display "raincloud plots" (half-violin plots + box plots + scatter plots) for all four categories… enabling an in-depth comparison of these two distinctly different distributions.

On this basis: - Set the overall canvas size to 14 inches wide × 10 inches high. - Continue using four fixed colors: light orange ‘#FFC0A0‘, light green ‘#B0E0B0‘, light purple ‘#B9A0E0‘, and beige ‘#FFE4C4‘. Use a red line to mark the mean value in the histograms. - Use a GridSpec layout with two rows and two columns. The first row spans both columns for the top plot, while the second row places the two histograms side by side, one in each column. The row height ratio should be explicitly set to 2:1. …. - Rotate the X-axis tick labels of the top subplot counterclockwise by 20 degrees. - Maintain a white background and gray grid lines (‘#D3D3D3‘).

Reference figure:

GT Code:

GT figure:

Instruction: Create a comprehensive, dashboard-style analytical view that juxtaposes raw data trends, statistical distributions, and localized details.

1. Layout Modifications: Use ‘GridSpec‘ to create a complex 2x2 grid layout. The top-left main plot (spanning the 1st row and 1st column) is a composite chart (three CCA lines + CKA bar chart). The top-right subplot (spanning the 1st row and 2nd column) is a box plot, used to display the overall data distribution of four data series (cca_top1, cca_top3, cca_top10, cka). The large bottom plot (spanning the 2nd row and all columns) is a "zoomed-in" view of the main plot, specifically focusing on the "Center Layer" in the range of 10 to 20 for the CCA line chart details.

2. Chart Type Conversion and Combination: In the top-right subplot, create a box plot for each of the four datasets and set appropriate labels. In the bottom zoomed-in plot, only draw the three CCA line charts and omit the CKA bar chart to emphasize the localized CCA dynamics. …

Additional Requirements: – Set the canvas size to 15×10 inches. – Use a 2×2 ‘GridSpec‘ layout with width ratios ‘[2,1]‘ and height ratios ‘[1,1]‘. The top-left main plot occupies the 1st row and 1st column, the top-right box plot occupies the 1st row and 2nd column, and the bottom zoomed-in plot spans the 2nd row across all columns… – For the box plots, use a fill color of ‘#d3d3d3‘, black borders, and red median lines. – For the zoomed-in region rectangle, use a gray fill with transparency 0.2, a red dashed border, and red dashed connecting lines.

Reference figure:

GT Code:

GT figure:

Instruction: Create a comprehensive, dashboard-style multi-panel analysis plot to deeply explore the relationships between model performance, tool wear growth, and model comparisons. The specific requirements are as follows:

Reference figure:

GT Code:

GT figure:

Instruction:

1. Use ‘GridSpec‘ to create a complex dashboard-style layout: - The left side contains a main plot occupying a 2x2 space. - The right side contains two subplots, each occupying a 1x1 space.

2. **Main Plot (Left Side)**: - Retain the original bar chart and exponential trend line. - Display the absolute values and trends of the annual research count.

3. **Top-Right Subplot**: - Convert the original data into an area chart. - Show the cumulative total of research counts to analyze the expansion of overall scale.

4. **Bottom-Right Subplot**: - Use a donut chart to display the proportion of research counts from the last three years (2022–2024) relative to their total. - Highlight the distribution of recent contributions.

5. Add titles to all subplots and ensure a unified visual style for clear communication and coordinated layout.

**Additional Modifications**: - Adjust the overall canvas size to 16 inches × 9 inches. - Configure the layout as ‘GridSpec(2,3)‘: - The main plot occupies the first and second columns of all rows. - The top-right subplot is placed in the first row, third column. - The bottom-right subplot is placed in the second row, third column. - **Styling**: - Main plot bar color: ‘’#1a5276’‘. - Main plot trend line color: ‘red‘. - Area chart fill color: ‘’#5dade2’‘, line color: ‘’#1a5276’‘. - Donut chart colors: ‘[’#1abc9c’, ’#f1c40f’, ’#e74c3c’]‘. - Donut chart percentage text: white and bold. - Overall title font: size 22, bold. - Subplot titles font: size 16. - Axis titles font: size 14. - Tick labels font: size 12. - Top-right chart annotations font: size 12, bold. - Donut chart center text font: size 14, bold. - Pie chart percentage text font: size 8, bold.

Reference figure:

GT Code:

GT figure:

Instruction:

Create a 2x2 dashboard to comprehensively compare model performance.

1. **Top-left plot (Performance Trend Comparison):** Divide the models into two groups: ’FinTabNet’ and ’PT1M-based’. …

2. **Top-right plot (Final Performance Ranking):** Use a horizontal bar chart to show the final accuracy of all 9 models at the last epoch..y.

3. **Bottom-left plot (Key Model Showdown):** Plot the performance curves of the best model ‘pt1m_av6‘ and the baseline model ‘pubtables‘ separately. Identify the epoch where ‘pt1m_av6‘ first surpasses ‘pubtables‘ by more than 0.05 in accuracy, and use ‘axvspan‘ to highlight the region from …

4. **Bottom-right plot (Performance vs. Stability):** Create a scatter plot where the X-axis represents the average accuracy of each model (mean over 30 epochs), and the Y-axis represents the standard deviation of accuracy. This plot evaluates whether high performance is accompanied by high instability. Add text labels to the best-performing, most stable, and most unstable models on the plot.

—Additional Modifications: - Set the overall canvas size to 16×12 inches. - Use a 2-row, 2-column ‘GridSpec‘ layout with row spacing of 0.4 and column spacing of 0.3. - Use a bold font size of 20 for the main title, regular font size of 12 for subplot titles, axis labels, and tick marks, and font size of 10 for legends… and semi-transparency. Use font size 9 for labels and adjust them horizontally by 0.002. - Use dashed grid lines with approximately 30

Reference figure:

GT Code:

GT figure:

Instruction: I have an Excel spreadsheet to analyze, which contains fuel types and corresponding horsepower values. Please generate a plotting code based on the style of the grouped box plot I uploaded to display the horsepower distribution for different fuel types. Use a canvas size precisely 13 inches wide and 8 inches high, with the color scheme set to Set3. The entire chart should contain only one subplot, without complex layouts like GridSpec. The title should be "Horsepower by Fuel Type," the X-axis label should be "Fuel Type," and the Y-axis label should be "Horsepower (hp)." Keep all text at Matplotlib’s default font size and style; rotate the X-axis tick labels 45 degrees; finally, apply a tight layout to ensure there is no excess whitespace between elements.

Reference Figure:

GT Code:

GT Figure:

Instruction: Based on the Excel table to be analyzed, mimic the drawing style of the image I uploaded as an attachment to create a scatter plot of Email1 length and Email2 length. The specific requirements are as follows: 1. Set the image size to 8 inches wide and 8 inches high; 2. Use cross-shaped markers for the scatter plot, with a fixed size of 200, a marker border width of 2, and map the "coolwarm" color scheme starting from sample index 1; 3. Add a color bar on the right side, with a gap of 0.05 between the color bar and the main plot, and set the aspect ratio to 1:30; 4. Add gray dashed arrows on the color bar, with the arrow style as "→", line type as dashed, line width of 2, pointing from above (2.8) to below (2.8) on the color bar scale; 5. Replace the color bar label with "Index", rotate it vertically by 90 degrees, font size 14, bold; 6. The main title of the chart is "(a) Correlation of Email1 and Email2 Lengths", font size 24, bold, 20 units from the top edge, with a vertical position set to 1.05; 7. Both the horizontal axis title "Email1 Length" and the vertical axis title "Email2 Length" should use font size 18, bold style, with a distance of 10 units from the axis labels; 8. Fix the axis range from 10 to 40, adjust the tick label font size to 14, and do not display grid lines.

Reference Figure:

GT Code:

GT Figure:

Instruction: I have an Excel spreadsheet to analyze, which contains two columns of data: “mental_health_history” and “depression.” I want to compare the distribution of depression scores between groups with and without a mental health history, mimicking the style of the image I uploaded as an attachment, and generate a box plot with a width of 10 inches and a height of 6 inches: - Use fill color "#FFA07A" for the group without a mental health history and "#20B2AA" for the group with a mental health history. The box edges, whiskers, caps, and median line colors should be "#CC8062" and "#1A8E88" (corresponding to the two groups). - Do not display outliers; - Plot scatter points offset by 0.2 on either side of the box, with scatter point colors matching the corresponding box fill color. The point edge color should be white, with an edge width of 0.5, size 50, opacity 0.8, and add random jitter of ±0.04 horizontally; - Set the overall background color to "#E5F7FD," grid line color to white, and style to solid lines; - X-axis tick labels should be "No History" and "With History," with a font size of 14; - Y-axis should display a range from 0 to 30 with a step of 5, and tick label font size should be 14; - Y-axis title should be "Depression Score," with a font size of 18 and bold; - Finally, call automatic layout adjustment to prevent label overlap.

Reference Figure:

GT Code:

GT Figure:

Figure 45: Selected charts of the Chart2Code.

21 Evaluation Code

21.1 color

Color evaluation code

21.2 Grid

21.3 Layout

21.4 Legend

21.5 Visual

21.6 Data

21.7 Text

21.8 Type

22 Prompt

22.1 generation Prompt

22.2 LLM-Score Prompt