Learn what matters: cross-domain imitation learning with task-relevant embeddings
This addresses the problem of training artificial agents from human demonstrations in different environments, offering a scalable solution without additional domain knowledge.
The paper tackles cross-domain imitation learning, enabling an agent to learn from demonstrations in a different domain without extra supervision, by using adversarial training and mutual information to learn task-relevant embeddings, and demonstrates successful policy transfer where other methods fail.
We study how an autonomous agent learns to perform a task from demonstrations in a different domain, such as a different environment or different agent. Such cross-domain imitation learning is required to, for example, train an artificial agent from demonstrations of a human expert. We propose a scalable framework that enables cross-domain imitation learning without access to additional demonstrations or further domain knowledge. We jointly train the learner agent's policy and learn a mapping between the learner and expert domains with adversarial training. We effect this by using a mutual information criterion to find an embedding of the expert's state space that contains task-relevant information and is invariant to domain specifics. This step significantly simplifies estimating the mapping between the learner and expert domains and hence facilitates end-to-end learning. We demonstrate successful transfer of policies between considerably different domains, without extra supervision such as additional demonstrations, and in situations where other methods fail.