Optimal Cycling of a Heterogenous Battery Bank via Reinforcement Learning
This work addresses battery degradation management for energy storage systems, presenting an incremental improvement through a tailored reinforcement learning approach.
The paper tackles the problem of minimizing long-term degradation costs in a heterogeneous battery bank with stochastic electricity generation and demand by formulating it as a Markov decision process and proposing a linear function approximation-based Q-learning algorithm with specially designed kernel functions.
We consider the problem of optimal charging/discharging of a bank of heterogenous battery units, driven by stochastic electricity generation and demand processes. The batteries in the battery bank may differ with respect to their capacities, ramp constraints, losses, as well as cycling costs. The goal is to minimize the degradation costs associated with battery cycling in the long run; this is posed formally as a Markov decision process. We propose a linear function approximation based Q-learning algorithm for learning the optimal solution, using a specially designed class of kernel functions that approximate the structure of the value functions associated with the MDP. The proposed algorithm is validated via an extensive case study.