SPRINT: Scalable Policy Pre-Training via Language Instruction Relabeling
This addresses the scalability issue in robot policy pre-training for household and kitchen tasks, offering an incremental improvement over prior methods.
The paper tackles the problem of reducing human annotation effort in pre-training robot policies by introducing SPRINT, which uses large language models for instruction relabeling and offline reinforcement learning for skill chaining, resulting in substantially faster learning of new long-horizon tasks in household and kitchen manipulation experiments.
Pre-training robot policies with a rich set of skills can substantially accelerate the learning of downstream tasks. Prior works have defined pre-training tasks via natural language instructions, but doing so requires tedious human annotation of hundreds of thousands of instructions. Thus, we propose SPRINT, a scalable offline policy pre-training approach which substantially reduces the human effort needed for pre-training a diverse set of skills. Our method uses two core ideas to automatically expand a base set of pre-training tasks: instruction relabeling via large language models and cross-trajectory skill chaining through offline reinforcement learning. As a result, SPRINT pre-training equips robots with a much richer repertoire of skills. Experimental results in a household simulator and on a real robot kitchen manipulation task show that SPRINT leads to substantially faster learning of new long-horizon tasks than previous pre-training approaches. Website at https://clvrai.com/sprint.