Build generally reusable agent-environment interaction models
This addresses the problem of transfer learning in reinforcement learning for researchers, though it appears incremental as it builds on existing successor feature and prototype concepts.
This paper tackles the problem of creating pre-trained models that are generally reusable for downstream reinforcement learning tasks, by proposing a method that builds agent-environment interaction models through domain-invariant successor features and behavior prototypes. Preliminary results show the approach can handle unseen changes in task objectives, environmental dynamics, and sensor modalities.
This paper tackles the problem of how to pre-train a model and make it generally reusable backbones for downstream task learning. In pre-training, we propose a method that builds an agent-environment interaction model by learning domain invariant successor features from the agent's vast experiences covering various tasks, then discretize them into behavior prototypes which result in an embodied set structure. To make the model generally reusable for downstream task learning, we propose (1) embodied feature projection that retains previous knowledge by projecting the new task's observation-action pair to the embodied set structure and (2) projected Bellman updates which add learning plasticity for the new task setting. We provide preliminary results that show downstream task learning based on a pre-trained embodied set structure can handle unseen changes in task objectives, environmental dynamics and sensor modalities.