Can Large Language Models Reason and Optimize Under Constraints?
This work identifies critical gaps in LLMs' ability to handle structured reasoning under constraints for real-world power grid optimization problems.
The paper investigated whether large language models can solve abstraction and optimization problems with constraints, specifically testing them on Optimal Power Flow problems, and found that state-of-the-art LLMs failed in most tasks and reasoning LLMs still failed in complex settings.
Large Language Models (LLMs) have demonstrated great capabilities across diverse natural language tasks; yet their ability to solve abstraction and optimization problems with constraints remains scarcely explored. In this paper, we investigate whether LLMs can reason and optimize under the physical and operational constraints of Optimal Power Flow (OPF) problem. We introduce a challenging evaluation setup that requires a set of fundamental skills such as reasoning, structured input handling, arithmetic, and constrained optimization. Our evaluation reveals that SoTA LLMs fail in most of the tasks, and that reasoning LLMs still fail in the most complex settings. Our findings highlight critical gaps in LLMs' ability to handle structured reasoning under constraints, and this work provides a rigorous testing environment for developing more capable LLM assistants that can tackle real-world power grid optimization problems.