Inductive Generalization in Reinforcement Learning from Specifications
This addresses the problem of sample inefficiency and generalization in reinforcement learning for tasks with inductive structures, though it appears incremental as it builds on existing RL from specifications methods.
The paper tackles the problem of generalizing reinforcement learning policies across inductive tasks with similar high-level goals but different low-level details, presenting a framework that learns a policy generator to create adapted policies for new task instances in a zero-shot manner. The result shows promise in generalizing to unseen policies for long-horizon tasks on challenging control benchmarks.
We present a novel inductive generalization framework for RL from logical specifications. Many interesting tasks in RL environments have a natural inductive structure. These inductive tasks have similar overarching goals but they differ inductively in low-level predicates and distributions. We present a generalization procedure that leverages this inductive relationship to learn a higher-order function, a policy generator, that generates appropriately adapted policies for instances of an inductive task in a zero-shot manner. An evaluation of the proposed approach on a set of challenging control benchmarks demonstrates the promise of our framework in generalizing to unseen policies for long-horizon tasks.