Transferring Agent Behaviors from Videos via Motion GANs
This addresses the bottleneck of reward specification in reinforcement learning for developing general agents, though it appears incremental as it builds on existing GAN and RL methods.
The paper tackles the problem of automatically specifying meaningful behaviors for reinforcement learning agents from raw pixels by training a generative adversarial network to produce motion templates as sub-goals, resulting in visually meaningful behaviors in unknown environments with novel agents.
A major bottleneck for developing general reinforcement learning agents is determining rewards that will yield desirable behaviors under various circumstances. We introduce a general mechanism for automatically specifying meaningful behaviors from raw pixels. In particular, we train a generative adversarial network to produce short sub-goals represented through motion templates. We demonstrate that this approach generates visually meaningful behaviors in unknown environments with novel agents and describe how these motions can be used to train reinforcement learning agents.