On-line reinforcement learning for optimization of real-life energy trading strategy
This addresses the challenge of energy market balancing for prosumers with renewable sources, though it is incremental as it applies existing RL methods to a specific domain.
The paper tackled the problem of automated day-ahead energy trading for a medium-sized prosumer by modeling it as a Markov Decision Process and optimizing a strategy using reinforcement learning algorithms, which generated the highest market profits compared to parametric strategies optimized with an evolutionary algorithm.
An increasing share of energy is produced from renewable sources by many small producers. The efficiency of those sources is volatile and, to some extent, random, exacerbating the problem of energy market balancing. In many countries, this balancing is done on the day-ahead (DA) energy markets. This paper considers automated trading on the DA energy market by a medium-sized prosumer. We model this activity as a Markov Decision Process and formalize a framework in which an applicable in real-life strategy can be optimized with off-line data. We design a trading strategy that is fed with the available environmental information that can impact future prices, including weather forecasts. We use state-of-the-art reinforcement learning (RL) algorithms to optimize this strategy. For comparison, we also synthesize simple parametric trading strategies and optimize them with an evolutionary algorithm. Results show that our RL-based strategy generates the highest market profits.