AMSbench: A Comprehensive Benchmark for Evaluating MLLM Capabilities in AMS Circuits
This addresses the problem of automating AMS circuit design for the IC industry by providing a systematic evaluation tool, though it is incremental as it builds on existing MLLM research without proposing a new method.
The authors tackled the lack of a comprehensive benchmark for evaluating Multi-modal Large Language Models (MLLMs) in Analog/Mixed-Signal (AMS) circuit design by introducing AMSbench, a suite with approximately 8000 test questions that revealed significant limitations in current models, particularly in complex reasoning and design tasks.
Analog/Mixed-Signal (AMS) circuits play a critical role in the integrated circuit (IC) industry. However, automating Analog/Mixed-Signal (AMS) circuit design has remained a longstanding challenge due to its difficulty and complexity. Although recent advances in Multi-modal Large Language Models (MLLMs) offer promising potential for supporting AMS circuit analysis and design, current research typically evaluates MLLMs on isolated tasks within the domain, lacking a comprehensive benchmark that systematically assesses model capabilities across diverse AMS-related challenges. To address this gap, we introduce AMSbench, a benchmark suite designed to evaluate MLLM performance across critical tasks including circuit schematic perception, circuit analysis, and circuit design. AMSbench comprises approximately 8000 test questions spanning multiple difficulty levels and assesses eight prominent models, encompassing both open-source and proprietary solutions such as Qwen 2.5-VL and Gemini 2.5 Pro. Our evaluation highlights significant limitations in current MLLMs, particularly in complex multi-modal reasoning and sophisticated circuit design tasks. These results underscore the necessity of advancing MLLMs' understanding and effective application of circuit-specific knowledge, thereby narrowing the existing performance gap relative to human expertise and moving toward fully automated AMS circuit design workflows. Our data is released at this URL.