Electronic Circuit Principles of Large Language Models
This provides a rigorous framework for predicting LLM performance and optimizing modular components, addressing a foundational challenge in AI interpretability and efficiency.
The paper tackled the problem of understanding and predicting the behavior of large language models (LLMs) by introducing Electronic Circuit Principles (ECP), which maps inference-time learning and reasoning to circuit analogies, resulting in a 60% improvement in Pearson correlation for task performance predictions and enabling new modular interventions that exceed median scores in competitive benchmarks.
Large language models (LLMs) such as DeepSeek-R1 have achieved remarkable performance across diverse reasoning tasks. To uncover the principles that govern their behaviour, we introduce the Electronic Circuit Principles (ECP), which maps inference-time learning (ITL) onto a semantic electromotive force and inference-time reasoning (ITR) onto a resistive network governed by Ohm's and Faraday's laws. This circuit-based modelling yields closed-form predictions of task performance and reveals how modular prompt components interact to shape accuracy. We validated ECP on 70,000 samples spanning 350 reasoning tasks and 9 advanced LLMs, observing a about 60% improvement in Pearson correlation relative to the conventional inference-time scaling law. Moreover, ECP explains the efficacy of 15 established prompting strategies and directs the development of new modular interventions that exceed the median score of the top 80% of participants in both the International Olympiad in Informatics and the International Mathematical Olympiad. By grounding LLM reasoning in electronic-circuit principles, ECP provides a rigorous framework for predicting performance and optimising modular components.