Signal Temporal Logic-Guided Apprenticeship Learning
This addresses the challenge of apprenticeship learning in robotics for tasks with temporal dependencies, offering a domain-specific improvement over prior methods.
The paper tackles the problem of learning control policies from demonstrations when tasks have temporal sub-goals, by using temporal logic specifications to improve reward inference, resulting in a drastic reduction in the number of demonstrations needed.
Apprenticeship learning crucially depends on effectively learning rewards, and hence control policies from user demonstrations. Of particular difficulty is the setting where the desired task consists of a number of sub-goals with temporal dependencies. The quality of inferred rewards and hence policies are typically limited by the quality of demonstrations, and poor inference of these can lead to undesirable outcomes. In this letter, we show how temporal logic specifications that describe high level task objectives, are encoded in a graph to define a temporal-based metric that reasons about behaviors of demonstrators and the learner agent to improve the quality of inferred rewards and policies. Through experiments on a diverse set of robot manipulator simulations, we show how our framework overcomes the drawbacks of prior literature by drastically improving the number of demonstrations required to learn a control policy.