Exchangeable Input Representations for Reinforcement Learning
This addresses a bottleneck in reinforcement learning for domains with unordered or variable inputs, offering a domain-specific improvement.
The paper tackles poor sample efficiency in deep reinforcement learning by introducing an attention-based method to create order-invariant input representations, reducing the input space by a factor of m! for m objects and enabling handling of variable object counts, with experiments showing improved sample efficiency and solving previously intractable problems.
Poor sample efficiency is a major limitation of deep reinforcement learning in many domains. This work presents an attention-based method to project neural network inputs into an efficient representation space that is invariant under changes to input ordering. We show that our proposed representation results in an input space that is a factor of $m!$ smaller for inputs of $m$ objects. We also show that our method is able to represent inputs over variable numbers of objects. Our experiments demonstrate improvements in sample efficiency for policy gradient methods on a variety of tasks. We show that our representation allows us to solve problems that are otherwise intractable when using naïve approaches.