PixL2R: Guiding Reinforcement Learning Using Natural Language by Mapping Pixels to Rewards
This addresses the challenge of making RL more applicable to complex problems by reducing the number of environment interactions needed, though it is incremental as it builds on prior language-guided approaches.
The paper tackles the problem of sample inefficiency in reinforcement learning, especially in sparse reward settings, by proposing a model that maps pixels to rewards using free-form natural language descriptions, resulting in significantly improved sample efficiency in robot manipulation tasks.
Reinforcement learning (RL), particularly in sparse reward settings, often requires prohibitively large numbers of interactions with the environment, thereby limiting its applicability to complex problems. To address this, several prior approaches have used natural language to guide the agent's exploration. However, these approaches typically operate on structured representations of the environment, and/or assume some structure in the natural language commands. In this work, we propose a model that directly maps pixels to rewards, given a free-form natural language description of the task, which can then be used for policy learning. Our experiments on the Meta-World robot manipulation domain show that language-based rewards significantly improves the sample efficiency of policy learning, both in sparse and dense reward settings.