Deep Reinforcement Learning for Real-Time Optimization of Pumps in Water Distribution Systems
This addresses the challenge of real-time optimization for water distribution system operators, though it is incremental as it applies an existing DRL method to a specific domain.
The paper tackles the problem of real-time pump control in water distribution systems, which is computationally intensive with conventional methods, by using deep reinforcement learning to achieve over 0.98 efficiency compared to baselines and a 2x speedup.
Real-time control of pumps can be an infeasible task in water distribution systems (WDSs) because the calculation to find the optimal pump speeds is resource-intensive. The computational need cannot be lowered even with the capabilities of smart water networks when conventional optimization techniques are used. Deep reinforcement learning (DRL) is presented here as a controller of pumps in two WDSs. An agent based on a dueling deep q-network is trained to maintain the pump speeds based on instantaneous nodal pressure data. General optimization techniques (e.g., Nelder-Mead method, differential evolution) serve as baselines. The total efficiency achieved by the DRL agent compared to the best performing baseline is above 0.98, whereas the speedup is around 2x compared to that. The main contribution of the presented approach is that the agent can run the pumps in real-time because it depends only on measurement data. If the WDS is replaced with a hydraulic simulation, the agent still outperforms conventional techniques in search speed.