Learning Transferable Concepts in Deep Reinforcement Learning
This addresses the challenge of generalization in artificial agents, though it appears incremental as it builds on existing representation learning approaches.
The paper tackles the problem of deep reinforcement learning agents being unable to reuse learned information across tasks by introducing a method to learn discrete representations of sensory inputs through self-supervision, which increases sample efficiency in both known and unknown tasks.
While humans and animals learn incrementally during their lifetimes and exploit their experience to solve new tasks, standard deep reinforcement learning methods specialize to solve only one task at a time. As a result, the information they acquire is hardly reusable in new situations. Here, we introduce a new perspective on the problem of leveraging prior knowledge to solve future tasks. We show that learning discrete representations of sensory inputs can provide a high-level abstraction that is common across multiple tasks, thus facilitating the transference of information. In particular, we show that it is possible to learn such representations by self-supervision, following an information theoretic approach. Our method is able to learn concepts in locomotive and optimal control tasks that increase the sample efficiency in both known and unknown tasks, opening a new path to endow artificial agents with generalization abilities.