A Framework for Few-Shot Policy Transfer through Observation Mapping and Behavior Cloning
This work addresses the challenge of reducing training time in physical robotics by enabling transfer learning between semantically dissimilar domains, which is incremental as it builds on existing sim2real and GAN-based methods.
The paper tackles the problem of expensive interaction costs in reinforcement learning for robotics by proposing a framework for few-shot policy transfer between domains with dissimilar tasks and properties, achieving successful behavior policy transfer with limited target interactions.
Despite recent progress in Reinforcement Learning for robotics applications, many tasks remain prohibitively difficult to solve because of the expensive interaction cost. Transfer learning helps reduce the training time in the target domain by transferring knowledge learned in a source domain. Sim2Real transfer helps transfer knowledge from a simulated robotic domain to a physical target domain. Knowledge transfer reduces the time required to train a task in the physical world, where the cost of interactions is high. However, most existing approaches assume exact correspondence in the task structure and the physical properties of the two domains. This work proposes a framework for Few-Shot Policy Transfer between two domains through Observation Mapping and Behavior Cloning. We use Generative Adversarial Networks (GANs) along with a cycle-consistency loss to map the observations between the source and target domains and later use this learned mapping to clone the successful source task behavior policy to the target domain. We observe successful behavior policy transfer with limited target task interactions and in cases where the source and target task are semantically dissimilar.