Claude Klöckl

Viktor Zobernig, Richard A. Saldanha, Jinke He et al.

The RangL project hosted by The Alan Turing Institute aims to encourage the wider uptake of reinforcement learning by supporting competitions relating to real-world dynamic decision problems. This article describes the reusable code repository developed by the RangL team and deployed for the 2022 Pathways to Net Zero Challenge, supported by the UK Net Zero Technology Centre. The winning solutions to this particular Challenge seek to optimize the UK's energy transition policy to net zero carbon emissions by 2050. The RangL repository includes an OpenAI Gym reinforcement learning environment and code that supports both submission to, and evaluation in, a remote instance of the open source EvalAI platform as well as all winning learning agent strategies. The repository is an illustrative example of RangL's capability to provide a reusable structure for future challenges.

Claude Klöckl, Katharina Gruber, Peter Regner et al.

Rinne et al. conduct an interesting analysis of the impact of wind turbine technology and land-use on wind power potentials, which allows profound insights into each factors contribution to overall potentials. The paper presents a detailed model of site-specific wind turbine investment cost (i.e. road- and grid access costs) complemented by a model used to estimate site-independent costs. We believe that propose a cutting edge model of site-specific investment costs. However, the site-independent cost model is flawed in our opinion. This flaw most likely does not impact the results presented in the paper, although we expect a considerable generalization error. Thus the application of the wind turbine cost model in other contexts may lead to unreasonable results. More generally, the derivation of the wind turbine cost model serves as an example of how applications of automated regression analysis can go wrong.

Christoph Graf, Viktor Zobernig, Johannes Schmidt et al.

We test the performance of deep deterministic policy gradient (DDPG), a deep reinforcement learning algorithm, able to handle continuous state and action spaces, to learn Nash equilibria in a setting where firms compete in prices. These algorithms are typically considered model-free because they do not require transition probability functions (as in e.g., Markov games) or predefined functional forms. Despite being model-free, a large set of parameters are utilized in various steps of the algorithm. These are e.g., learning rates, memory buffers, state-space dimensioning, normalizations, or noise decay rates and the purpose of this work is to systematically test the effect of these parameter configurations on convergence to the analytically derived Bertrand equilibrium. We find parameter choices that can reach convergence rates of up to 99%. The reliable convergence may make the method a useful tool to study strategic behavior of firms even in more complex settings. Keywords: Bertrand Equilibrium, Competition in Uniform Price Auctions, Deep Deterministic Policy Gradient Algorithm, Parameter Sensitivity Analysis