A Hierarchical MARL-Based Approach for Coordinated Retail P2P Trading and Wholesale Market Participation of DERs
This work addresses the problem of grid operational flexibility and market efficiency for the electric energy sector, representing an incremental advancement in applying MARL to DER coordination.
The paper tackles the challenge of enabling distributed energy resources (DERs) to participate effectively in electricity markets by proposing a hierarchical multi-agent deep reinforcement learning (MARL) framework that coordinates peer-to-peer retail auctions and wholesale market aggregation, resulting in enhanced market performance.
The ongoing shift towards decentralization of the electric energy sector, driven by the growing electrification across end-use sectors, and widespread adoption of distributed energy resources (DERs), necessitates their active participation in the electricity markets to support grid operations. Furthermore, with bi-directional energy and communication flows becoming standard, intelligent, easy-to-deploy, resource-conservative demand-side participation is expected to play a critical role in securing power grid operational flexibility and market efficiency. This work proposes a market engagement framework that leverages a hierarchical multi-agent deep reinforcement learning (MARL) approach to enable individual prosumers to participate in peer-to-peer retail auctions and further aggregate these intelligent prosumers to facilitate effective DER participation in wholesale markets. Ultimately, a Stackelberg game is proposed to coordinate this hierarchical MARL-based DER market participation framework toward enhanced market performance.