Thomas Morstyn

Thomas Morstyn, Branislav Hredzak, Ricardo P. Aguilera et al.

This paper proposes a new convex model predictive control strategy for dynamic optimal power flow between battery energy storage systems distributed in an AC microgrid. The proposed control strategy uses a new problem formulation, based on a linear d-q reference frame voltage-current model and linearised power flow approximations. This allows the optimal power flows to be solved as a convex optimisation problem, for which fast and robust solvers exist. The proposed method does not assume real and reactive power flows are decoupled, allowing line losses, voltage constraints and converter current constraints to be addressed. In addition, non-linear variations in the charge and discharge efficiencies of lithium ion batteries are analysed and included in the control strategy. Real-time digital simulations were carried out for an islanded microgrid based on the IEEE 13 bus prototypical feeder, with distributed battery energy storage systems and intermittent photovoltaic generation. It is shown that the proposed control strategy approaches the performance of a strategy based on non-convex optimisation, while reducing the required computation time by a factor of 1000, making it suitable for a real-time model predictive control implementation.

Matthew Deakin, Thomas Morstyn, Dimitra Apostolopoulou et al.

Reactive power has been proposed as a method of voltage control for distribution networks, providing a means of increasing the amount of energy transferred from distributed generators to the bulk transmission network. The value of reactive power can therefore be measured according to an increase in transferred energy, where the transferred energy is defined as the total generated energy, less the total network losses. If network losses are ignored, an error in the valuation of a given amount of reactive power will be observed (leading to reactive power provision being under- or over-valued). The non-linear analytic solution of a two-bus network is studied, and non-trivial upper and lower bounds are determined for this `valuation error'. The properties predicted by this two-bus network are demonstrated to hold on a three-phase unbalanced distribution test feeder with good accuracy. This allows for an analytic assessment of the importance of losses in the valuation of reactive power in arbitrary networks.

Flora Charbonnier, Thomas Morstyn, Malcolm McCulloch

In this paper, we present the Home Electricity Data Generator (HEDGE), an open-access tool for the random generation of realistic residential energy data. HEDGE generates realistic daily profiles of residential PV generation, household electric loads, and electric vehicle consumption and at-home availability, based on real-life UK datasets. The lack of usable data is a major hurdle for research on residential distributed energy resources characterisation and coordination, especially when using data-driven methods such as machine learning-based forecasting and reinforcement learning-based control. A key issue is that while large data banks are available, they are not in a usable format, and numerous subsequent days of data for a given single home are unavailable. We fill these gaps with the open-access HEDGE tool which generates data sequences of energy data for several days in a way that is consistent for single homes, both in terms of profile magnitude and behavioural clusters. From raw datasets, pre-processing steps are conducted, including filling in incomplete data sequences and clustering profiles into behaviour clusters. Generative adversarial networks (GANs) are then trained to generate realistic synthetic data representative of each behaviour groups consistent with real-life behavioural and physical patterns.

Matthew Deakin, Thomas Morstyn, Dimitra Apostolopoulou et al.

In this paper, the power flow solution of the two bus network is used to analytically characterise maximum power transfer limits of distribution networks, when subject to both thermal and voltage constraints. Traditional analytic methods are shown to reach contradictory conclusions on the suitability of reactive power for increasing power transfer. Therefore, a more rigorous analysis is undertaken, yielding two solutions, both fully characterised by losses. The first is the well-known thermal limit. The second we define as the `marginal loss-induced maximum power transfer limit'. This is a point at which the marginal increases in losses are greater than increases in generated power. The solution is parametrised in terms of the ratio of resistive to reactive impedance, and yields the reactive power required. The accuracy and existence of these solutions are investigated using the IEEE 34 bus distribution test feeder, and show good agreement with the two bus approximation. The work has implications for the analysis of reactive power interventions in distribution networks, and for the optimal sizing of distributed generation.

Yihong Zhou, Dylan Cope, Jakob Foerster et al.

Coordinating growing grid flexibility under uncertainty is becoming increasingly important for efficient and reliable power-system operation. A core computational requirement is the efficient large-scale batched evaluation of AC power flow across candidate operating actions and uncertainty scenarios. Previous work has explored GPU-based batched power-flow evaluation, but has largely relied on hand-written C or CUDA code, creating barriers to customisation, efficient kernel optimisation, and long-term maintenance. JAX is a Python-based framework that enables efficient accelerator execution while keeping implementations in Python. This letter therefore proposes a JAX-based batched AC power-flow solver that uses current JAX functionality to implement Newton--Raphson for transmission networks and Z-Bus power flow for three-phase unbalanced distribution networks, achieving more than 10x speed-ups relative to pandapower and OpenDSS. In addition, JAX integrates seamlessly with the broader JAX-based AI ecosystem, making it straightforward to embed power-flow evaluation within AI methods for future larger-scale and more complex power-system operation.

Yihong Zhou, Hongtai Zeng, Thomas Morstyn

Coordinating large populations of grid-edge devices requires learning methods that remain fully decentralised in deployment while still respecting three-phase AC distribution-network physics. This paper proposes gradient-based multi-agent proximal learning (GradMAP) to address this challenge. GradMAP trains independent neural-network policies for each agent without any parameter sharing, and each agent uses only its own local observation for online decision-making without communication. During offline training, GradMAP embeds a differentiable three-phase AC power-flow model in a primal-dual learning loop and uses implicit differentiation to propagate exact network-constraint violations to update the policy parameters. To speed up training, GradMAP reuses expensive environment gradients through a proximal surrogate within a trust region defined in the more direct policy-output (action) space, instead of the probability distribution space used in other works, such as PPO. In case studies with 1,000 agents managing batteries, heat pumps, and controllable generators on the IEEE 123-bus feeder, GradMAP learns decentralised policies that minimise three-phase AC load-flow constraint violations within 15 minutes of training on a single workstation-class NVIDIA RTX PRO 5000 Blackwell 48GB GPU. This is a 3--5x training speed-up over gradient-based self-supervised learning benchmarks and substantially better training efficiency than multi-agent reinforcement-learning benchmarks. In out-of-sample tests, GradMAP also delivers among the lowest operating cost and constraint violations.

François Michelon, Yihong Zhou, Thomas Morstyn

Home Energy Management Systems (HEMSs) help households tailor their electricity usage based on power system signals such as energy prices. This technology helps to reduce energy bills and offers greater demand-side flexibility that supports the power system stability. However, residents who lack a technical background may find it difficult to use HEMSs effectively, because HEMSs require well-formatted parameterization that reflects the characteristics of the energy resources, houses, and users' needs. Recently, Large-Language Models (LLMs) have demonstrated an outstanding ability in language understanding. Motivated by this, we propose an LLM-based interface that interacts with users to understand and parameterize their ``badly-formatted answers'', and then outputs well-formatted parameters to implement an HEMS. We further use Reason and Act method (ReAct) and few-shot prompting to enhance the LLM performance. Evaluating the interface performance requires multiple user--LLM interactions. To avoid the efforts in finding volunteer users and reduce the evaluation time, we additionally propose a method that uses another LLM to simulate users with varying expertise, ranging from knowledgeable to non-technical. By comprehensive evaluation, the proposed LLM-based HEMS interface achieves an average parameter retrieval accuracy of 88\%, outperforming benchmark models without ReAct and/or few-shot prompting.

Sarah Keren, Chaimaa Essayeh, Stefano V. Albrecht et al.

The rapidly changing architecture and functionality of electrical networks and the increasing penetration of renewable and distributed energy resources have resulted in various technological and managerial challenges. These have rendered traditional centralized energy-market paradigms insufficient due to their inability to support the dynamic and evolving nature of the network. This survey explores how multi-agent reinforcement learning (MARL) can support the decentralization and decarbonization of energy networks and mitigate the associated challenges. This is achieved by specifying key computational challenges in managing energy networks, reviewing recent research progress on addressing them, and highlighting open challenges that may be addressed using MARL.

Flora Charbonnier, Bei Peng, Thomas Morstyn et al.

This paper investigates how deep multi-agent reinforcement learning can enable the scalable and privacy-preserving coordination of residential energy flexibility. The coordination of distributed resources such as electric vehicles and heating will be critical to the successful integration of large shares of renewable energy in our electricity grid and, thus, to help mitigate climate change. The pre-learning of individual reinforcement learning policies can enable distributed control with no sharing of personal data required during execution. However, previous approaches for multi-agent reinforcement learning-based distributed energy resources coordination impose an ever greater training computational burden as the size of the system increases. We therefore adopt a deep multi-agent actor-critic method which uses a \emph{centralised but factored critic} to rehearse coordination ahead of execution. Results show that coordination is achieved at scale, with minimal information and communication infrastructure requirements, no interference with daily activities, and privacy protection. Significant savings are obtained for energy users, the distribution network and greenhouse gas emissions. Moreover, training times are nearly 40 times shorter than with a previous state-of-the-art reinforcement learning approach without the factored critic for 30 homes.

Liyang Han, Thomas Morstyn, Constance Crozier et al.

Among the various market structures under peer-to-peer energy sharing, one model based on cooperative game theory provides clear incentives for prosumers to collaboratively schedule their energy resources. The computational complexity of this model, however, increases exponentially with the number of participants. To address this issue, this paper proposes the application of K-means clustering to the energy profiles following the grand coalition optimization. The cooperative model is run with the "clustered players" to compute their payoff allocations, which are then further distributed among the prosumers within each cluster. Case studies show that the proposed method can significantly improve the scalability of the cooperative scheme while maintaining a high level of financial incentives for the prosumers.