IR-VIC: Unsupervised Discovery of Sub-goals for Transfer in RL
This addresses the challenge of efficient exploration in reinforcement learning for researchers and practitioners, though it appears incremental as it builds on existing variational intrinsic control methods.
The paper tackles the problem of identifying sub-goals for exploration in partially observable sequential decision-making tasks, proposing an unsupervised framework that achieves better exploration and sample complexity on challenging grid-world navigation tasks compared to supervised methods.
We propose a novel framework to identify sub-goals useful for exploration in sequential decision making tasks under partial observability. We utilize the variational intrinsic control framework (Gregor et.al., 2016) which maximizes empowerment -- the ability to reliably reach a diverse set of states and show how to identify sub-goals as states with high necessary option information through an information theoretic regularizer. Despite being discovered without explicit goal supervision, our sub-goals provide better exploration and sample complexity on challenging grid-world navigation tasks compared to supervised counterparts in prior work.