A Composable Specification Language for Reinforcement Learning Tasks
This addresses the problem of manual reward design for robot control tasks in reinforcement learning, representing an incremental improvement in specification methods.
The authors tackled the challenge of specifying complex reinforcement learning tasks with multiple objectives and safety constraints by proposing a language and algorithm that automatically compiles specifications into reward functions and performs reward shaping, showing that their tool SPECTRL outperforms state-of-the-art baselines.
Reinforcement learning is a promising approach for learning control policies for robot tasks. However, specifying complex tasks (e.g., with multiple objectives and safety constraints) can be challenging, since the user must design a reward function that encodes the entire task. Furthermore, the user often needs to manually shape the reward to ensure convergence of the learning algorithm. We propose a language for specifying complex control tasks, along with an algorithm that compiles specifications in our language into a reward function and automatically performs reward shaping. We implement our approach in a tool called SPECTRL, and show that it outperforms several state-of-the-art baselines.