Efficient Representation for Electric Vehicle Charging Station Operations using Reinforcement Learning
This work addresses operational efficiency for electric vehicle charging stations, but it is incremental as it builds on existing reinforcement learning techniques with specific adaptations.
The paper tackled the high-dimensional and time-varying state/action space problem in reinforcement learning for electric vehicle charging station operations by developing aggregation schemes based on laxity values, resulting in higher rewards and more effective policies compared to existing methods.
Effectively operating electrical vehicle charging station (EVCS) is crucial for enabling the rapid transition of electrified transportation. To solve this problem using reinforcement learning (RL), the dimension of state/action spaces scales with the number of EVs and is thus very large and time-varying. This dimensionality issue affects the efficiency and convergence properties of generic RL algorithms. We develop aggregation schemes that are based on the emergency of EV charging, namely the laxity value. A least-laxity first (LLF) rule is adopted to consider only the total charging power of the EVCS which ensures the feasibility of individual EV schedules. In addition, we propose an equivalent state aggregation that can guarantee to attain the same optimal policy. Based on the proposed representation, policy gradient method is used to find the best parameters for the linear Gaussian policy . Numerical results have validated the performance improvement of the proposed representation approaches in attaining higher rewards and more effective policies as compared to existing approximation based approach.