Zero-Shot Policy Transfer with Disentangled Task Representation of Meta-Reinforcement Learning
This addresses the problem of rapid adaptation in RL agents for simulated and real-world robotic tasks, though it appears incremental as it builds on existing meta-RL and compositionality concepts.
The paper tackled zero-shot policy generalization in reinforcement learning by using a meta-RL algorithm with disentangled task representations, achieving policy generalization to unseen compositional tasks without extra exploration.
Humans are capable of abstracting various tasks as different combinations of multiple attributes. This perspective of compositionality is vital for human rapid learning and adaption since previous experiences from related tasks can be combined to generalize across novel compositional settings. In this work, we aim to achieve zero-shot policy generalization of Reinforcement Learning (RL) agents by leveraging the task compositionality. Our proposed method is a meta- RL algorithm with disentangled task representation, explicitly encoding different aspects of the tasks. Policy generalization is then performed by inferring unseen compositional task representations via the obtained disentanglement without extra exploration. The evaluation is conducted on three simulated tasks and a challenging real-world robotic insertion task. Experimental results demonstrate that our proposed method achieves policy generalization to unseen compositional tasks in a zero-shot manner.