SpecVQA: A Benchmark for Spectral Understanding and Visual Question Answering in Scientific Images
This benchmark addresses the lack of evaluation for MLLMs on scientific spectral understanding, which is crucial for advancing AI in scientific data analysis.
SpecVQA introduces a benchmark for evaluating multimodal large language models on scientific spectral understanding, covering 7 spectrum types with 620 figures and 3100 QA pairs. The proposed spectral data sampling and interpolation reconstruction approach improves performance on the benchmark.
Spectra are a prevalent yet highly information-dense form of scientific imagery, presenting substantial challenges to multimodal large language models (MLLMs) due to their unstructured and domain-specific characteristics. Here we introduce SpecVQA, a professional scientific-image benchmark for evaluating multimodal models on scientific spectral understanding, covering 7 representative spectrum types with expert-annotated question-answer pairs. The aim comprises two aspects: spectra scientific QA evaluation and corresponding underlying task evaluation. SpecVQA contains 620 figures and 3100 QA pairs curated from peer-reviewed literature, targeting both direct information extraction and domain-specific reasoning. To effectively reduce token length while preserving essential curve characteristics, we propose a spectral data sampling and interpolation reconstruction approach. Ablation studies further confirm that the approach achieves substantial performance improvements on the proposed benchmark. We test the capability of prominent MLLMs in scientific spectral understanding on our benchmark and present a leaderboard. This work represents an essential step toward enhancing spectral understanding in multimodal large models and suggests promising directions for extending visual-language models to broader scientific research and data analysis.