Federated Reinforcement Learning for Electric Vehicles Charging Control on Distribution Networks
This addresses grid stability and driver privacy issues for electric vehicle integration, representing an incremental improvement over existing multi-agent deep reinforcement learning methods.
The paper tackles EV charging control by integrating multi-EV charging/discharging with a radial distribution network under optimal power flow, proposing a federated deep reinforcement learning algorithm (FedSAC) that balances V2G profits, grid load, and driver anxiety, with simulation results showing effectiveness in strategy diversity, power fluctuation reduction, convergence efficiency, and generalization.
With the growing popularity of electric vehicles (EVs), maintaining power grid stability has become a significant challenge. To address this issue, EV charging control strategies have been developed to manage the switch between vehicle-to-grid (V2G) and grid-to-vehicle (G2V) modes for EVs. In this context, multi-agent deep reinforcement learning (MADRL) has proven its effectiveness in EV charging control. However, existing MADRL-based approaches fail to consider the natural power flow of EV charging/discharging in the distribution network and ignore driver privacy. To deal with these problems, this paper proposes a novel approach that combines multi-EV charging/discharging with a radial distribution network (RDN) operating under optimal power flow (OPF) to distribute power flow in real time. A mathematical model is developed to describe the RDN load. The EV charging control problem is formulated as a Markov Decision Process (MDP) to find an optimal charging control strategy that balances V2G profits, RDN load, and driver anxiety. To effectively learn the optimal EV charging control strategy, a federated deep reinforcement learning algorithm named FedSAC is further proposed. Comprehensive simulation results demonstrate the effectiveness and superiority of our proposed algorithm in terms of the diversity of the charging control strategy, the power fluctuations on RDN, the convergence efficiency, and the generalization ability.