Deep Sets for Generalization in RL
This addresses generalization challenges in RL for agents interacting with objects based on natural language goals, though it appears incremental as it builds on existing Deep Sets and attention methods.
The paper tackles the problem of generalization in language-guided reinforcement learning by encoding object-centered representations in reward functions and policy architectures using Deep Sets and gated-attention mechanisms, showing strong generalization to out-of-distribution goals in a 2D procedurally-generated world with concrete results on varying object numbers and relational reasoning.
This paper investigates the idea of encoding object-centered representations in the design of the reward function and policy architectures of a language-guided reinforcement learning agent. This is done using a combination of object-wise permutation invariant networks inspired from Deep Sets and gated-attention mechanisms. In a 2D procedurally-generated world where agents targeting goals in natural language navigate and interact with objects, we show that these architectures demonstrate strong generalization capacities to out-of-distribution goals. We study the generalization to varying numbers of objects at test time and further extend the object-centered architectures to goals involving relational reasoning.