From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Following
This addresses the problem of enabling robots to follow natural language instructions more effectively, though it is incremental as it builds on existing inverse reinforcement learning methods.
The paper tackles the challenge of specifying goals for autonomous machines by grounding language commands as reward functions using inverse reinforcement learning, demonstrating that this approach outperforms directly learning language-conditioned policies in novel tasks and environments with high-dimensional visual inputs.
Reinforcement learning is a promising framework for solving control problems, but its use in practical situations is hampered by the fact that reward functions are often difficult to engineer. Specifying goals and tasks for autonomous machines, such as robots, is a significant challenge: conventionally, reward functions and goal states have been used to communicate objectives. But people can communicate objectives to each other simply by describing or demonstrating them. How can we build learning algorithms that will allow us to tell machines what we want them to do? In this work, we investigate the problem of grounding language commands as reward functions using inverse reinforcement learning, and argue that language-conditioned rewards are more transferable than language-conditioned policies to new environments. We propose language-conditioned reward learning (LC-RL), which grounds language commands as a reward function represented by a deep neural network. We demonstrate that our model learns rewards that transfer to novel tasks and environments on realistic, high-dimensional visual environments with natural language commands, whereas directly learning a language-conditioned policy leads to poor performance.