Enabling Large Language Models to Perform Power System Simulations with Previously Unseen Tools: A Case of Daline
This addresses the problem of LLMs lacking knowledge for complex power grid simulations, offering a domain-specific solution for researchers in power systems.
The paper tackled the challenge of enabling large language models (LLMs) to perform power system simulations with previously unseen tools, proposing a modular framework that improved GPT-4o's simulation coding accuracy from 0% to 96.07% on tasks in the Daline toolbox.
The integration of experiment technologies with large language models (LLMs) is transforming scientific research, offering AI capabilities beyond specialized problem-solving to becoming research assistants for human scientists. In power systems, simulations are essential for research. However, LLMs face significant challenges in power system simulations due to limited pre-existing knowledge and the complexity of power grids. To address this issue, this work proposes a modular framework that integrates expertise from both the power system and LLM domains. This framework enhances LLMs' ability to perform power system simulations on previously unseen tools. Validated using 34 simulation tasks in Daline, a (optimal) power flow simulation and linearization toolbox not yet exposed to LLMs, the proposed framework improved GPT-4o's simulation coding accuracy from 0% to 96.07%, also outperforming the ChatGPT-4o web interface's 33.8% accuracy (with the entire knowledge base uploaded). These results highlight the potential of LLMs as research assistants in power systems.