Transferring Multiple Policies to Hotstart Reinforcement Learning in an Air Compressor Management Problem
This work addresses the need for quick, resource-efficient controller deployment in industrial settings like air compressor management, though it is incremental as it builds on existing policy shaping methods.
The paper tackles the problem of efficiently training reinforcement learning controllers for new industrial machines by transferring knowledge from existing controllers, showing that their approach outperforms loading an old controller and significantly improves long-term performance.
Many instances of similar or almost-identical industrial machines or tools are often deployed at once, or in quick succession. For instance, a particular model of air compressor may be installed at hundreds of customers. Because these tools perform distinct but highly similar tasks, it is interesting to be able to quickly produce a high-quality controller for machine $N+1$ given the controllers already produced for machines $1..N$. This is even more important when the controllers are learned through Reinforcement Learning, as training takes time, energy and other resources. In this paper, we apply Policy Intersection, a Policy Shaping method, to help a Reinforcement Learning agent learn to solve a new variant of a compressors control problem faster, by transferring knowledge from several previously learned controllers. We show that our approach outperforms loading an old controller, and significantly improves performance in the long run.