Inversely Learning Transferable Rewards via Abstracted States
This work addresses the challenge of integrating robots into new processing lines with shared intrinsic preferences, offering a transferable solution for robotic applications.
The paper tackles the problem of learning transferable intrinsic reward functions from behavior trajectories in multiple domain instances, enabling robots to adapt to new tasks without reprogramming. The method successfully learns abstract reward functions that produce task behaviors in unseen instances across OpenAI Gym and AssistiveGym domains.
Inverse reinforcement learning (IRL) has progressed significantly toward accurately learning the underlying rewards in both discrete and continuous domains from behavior data. The next advance is to learn {\em intrinsic} preferences in ways that produce useful behavior in settings or tasks which are different but aligned with the observed ones. In the context of robotic applications, this helps integrate robots into processing lines involving new tasks (with shared intrinsic preferences) without programming from scratch. We introduce a method to inversely learn an abstract reward function from behavior trajectories in two or more differing instances of a domain. The abstract reward function is then used to learn task behavior in another separate instance of the domain. This step offers evidence of its transferability and validates its correctness. We evaluate the method on trajectories in tasks from multiple domains in OpenAI's Gym testbed and AssistiveGym and show that the learned abstract reward functions can successfully learn task behaviors in instances of the respective domains, which have not been seen previously.