Grounding LTL Tasks in Sub-Symbolic RL Environments for Zero-Shot Generalization
This addresses the challenge of zero-shot generalization in RL for temporally-extended tasks, though it is incremental by building on Neural Reward Machines.
The paper tackles the problem of training a reinforcement learning agent to follow multiple Linear Temporal Logic instructions in sub-symbolic environments without prior knowledge of symbol mappings, achieving performance comparable to using true symbol grounding and significantly outperforming state-of-the-art methods.
In this work we address the problem of training a Reinforcement Learning agent to follow multiple temporally-extended instructions expressed in Linear Temporal Logic in sub-symbolic environments. Previous multi-task work has mostly relied on knowledge of the mapping between raw observations and symbols appearing in the formulae. We drop this unrealistic assumption by jointly training a multi-task policy and a symbol grounder with the same experience. The symbol grounder is trained only from raw observations and sparse rewards via Neural Reward Machines in a semi-supervised fashion. Experiments on vision-based environments show that our method achieves performance comparable to using the true symbol grounding and significantly outperforms state-of-the-art methods for sub-symbolic environments.