Human-like Energy Management Based on Deep Reinforcement Learning and Historical Driving Experiences
This work addresses the need for efficient and real-time energy management in hybrid electric vehicles, representing an incremental improvement by integrating expert knowledge and historical data into a reinforcement learning approach.
The paper tackles the problem of developing an energy management strategy for hybrid electric vehicles by proposing a human-like framework that combines deep reinforcement learning with historical driving data, resulting in improved fuel economy and convergence rate.
Development of hybrid electric vehicles depends on an advanced and efficient energy management strategy (EMS). With online and real-time requirements in mind, this article presents a human-like energy management framework for hybrid electric vehicles according to deep reinforcement learning methods and collected historical driving data. The hybrid powertrain studied has a series-parallel topology, and its control-oriented modeling is founded first. Then, the distinctive deep reinforcement learning (DRL) algorithm, named deep deterministic policy gradient (DDPG), is introduced. To enhance the derived power split controls in the DRL framework, the global optimal control trajectories obtained from dynamic programming (DP) are regarded as expert knowledge to train the DDPG model. This operation guarantees the optimality of the proposed control architecture. Moreover, the collected historical driving data based on experienced drivers are employed to replace the DP-based controls, and thus construct the human-like EMSs. Finally, different categories of experiments are executed to estimate the optimality and adaptability of the proposed human-like EMS. Improvements in fuel economy and convergence rate indicate the effectiveness of the constructed control structure.