Adaptable Automation with Modular Deep Reinforcement Learning and Policy Transfer
This work aims to improve the adaptability and efficiency of deep RL agents for industrial automation, which currently suffer from narrow task specialization and sample inefficiency.
This paper addresses the limitations of deep Reinforcement Learning (RL) algorithms in industrial adoption by developing a Hyper-Actor Soft Actor-Critic (HASAC) RL framework. The HASAC framework, which uses task modularization and transfer learning, demonstrated superior performance in reward value, success rate, and task completion time compared to state-of-the-art deep RL algorithms on the Meta-World robotic manipulation benchmark.
Recent advances in deep Reinforcement Learning (RL) have created unprecedented opportunities for intelligent automation, where a machine can autonomously learn an optimal policy for performing a given task. However, current deep RL algorithms predominantly specialize in a narrow range of tasks, are sample inefficient, and lack sufficient stability, which in turn hinder their industrial adoption. This article tackles this limitation by developing and testing a Hyper-Actor Soft Actor-Critic (HASAC) RL framework based on the notions of task modularization and transfer learning. The goal of the proposed HASAC is to enhance the adaptability of an agent to new tasks by transferring the learned policies of former tasks to the new task via a "hyper-actor". The HASAC framework is tested on a new virtual robotic manipulation benchmark, Meta-World. Numerical experiments show superior performance by HASAC over state-of-the-art deep RL algorithms in terms of reward value, success rate, and task completion time.