Accelerating Reinforcement Learning for Wind Farm Control via Expert Demonstrations
For wind farm operators, this method reduces the costly initial learning phase of RL-based control, enabling faster deployment with minimal performance loss.
Reinforcement learning for wind farm control suffers from slow convergence and poor initial performance. By pretraining with expert demonstrations from a steady-state wake model, initial performance was raised from 12% below baseline to near-baseline levels, and all configurations converged within 250,000 steps, ultimately exceeding a lookup-table controller's 7% power gain.
Reinforcement learning (RL) offers a promising approach for adaptive wind farm flow control, yet its practical deployment is hindered by slow training convergence and poor initial performance, factors that could translate to years of reduced power output if an untrained agent were deployed directly. This work investigates whether domain knowledge from steady-state wake models can accelerate RL training and improve initial controller performance. We propose a pretraining methodology in which expert demonstrations are generated by deploying a PyWake-based steady-state optimizer within a dynamic wake simulation (WindGym), then used to initialize both the actor and critic networks of a Soft Actor-Critic agent via behavior cloning. Experiments on a 2x2 wind farm show that pretraining eliminates the costly initial learning phase: while an untrained agent underperforms the greedy zero-yaw baseline by approximately 12%, pretraining raises initial performance to near-baseline levels. During online fine-tuning, all configurations converge within 250,000 environment steps to achieve similar performance, ultimately exceeding that of a lookup-table controller, which reaches approximately 7% power gain after 500,000 steps.