The Pump Scheduling Problem: A Real-World Scenario for Reinforcement Learning
This provides a new benchmark for researchers in reinforcement learning to address challenges like safety constraints and partial observability in real-world applications, though it is incremental as it adapts existing methods to a specific domain.
The authors tackled the lack of complex real-world benchmarks for deep reinforcement learning by introducing a testbed based on the pump scheduling problem in a water distribution facility, which includes a realistic simulator, three years of high-resolution operational data, and a baseline RL formulation.
Deep Reinforcement Learning (DRL) has demonstrated impressive results in domains such as games and robotics, where task formulations are well-defined. However, few DRL benchmarks are grounded in complex, real-world environments, where safety constraints, partial observability, and the need for hand-engineered task representations pose significant challenges. To help bridge this gap, we introduce a testbed based on the pump scheduling problem in a real-world water distribution facility. The task involves controlling pumps to ensure a reliable water supply while minimizing energy consumption and respecting the constraints of the system. Our testbed includes a realistic simulator, three years of high-resolution (1-minute) operational data from human-led control, and a baseline RL task formulation. This testbed supports a wide range of research directions, including offline RL, safe exploration, inverse RL, and multi-objective optimization.