Scientific Reasoning: Assessment of Multimodal Generative LLMs
This work assesses the scientific reasoning capabilities of multimodal LLMs, providing benchmarks for researchers and developers, but it is incremental as it focuses on evaluation rather than introducing new methods.
The study evaluated multimodal LLMs on ScienceQA, finding Gemini models achieved the highest accuracy with minimal context and the greatest textual similarity to human explanations with more context, while adapter-tuning and training from Gemini outputs did not improve performance reliably.
Large language models (LLMs) can answer questions and reason about complex tasks, also from the scientific domain. We assess several multimodal LLMs (MLLMs) on ScienceQA and find that Gemini models show the highest accuracy with little context, and the highest textual similarity to human explanations with richer context. Adapter-tuning of smaller MLLMs did not lead to any reliable performance. Training from Gemini outputs consistently underperformed training from the original data.