Refactoring Policy for Compositional Generalizability using Self-Supervised Object Proposals
This addresses the challenge of compositional generalization in reinforcement learning, which is important for developing more robust AI agents, though it appears incremental as it builds on existing teacher-student and object-centric methods.
The paper tackles the problem of learning policies with compositional generalizability by proposing a two-stage framework that refactorizes a teacher policy into a student policy with object-centric inductive bias, achieving superior performance on four difficult tasks compared to baselines.
We study how to learn a policy with compositional generalizability. We propose a two-stage framework, which refactorizes a high-reward teacher policy into a generalizable student policy with strong inductive bias. Particularly, we implement an object-centric GNN-based student policy, whose input objects are learned from images through self-supervised learning. Empirically, we evaluate our approach on four difficult tasks that require compositional generalizability, and achieve superior performance compared to baselines.