Object Exchangeability in Reinforcement Learning: Extended Abstract
This work addresses sample efficiency for reinforcement learning practitioners, offering a novel representation to handle object exchangeability, though it is incremental as it builds on existing attention mechanisms.
The paper tackles the problem of sample inefficiency in deep reinforcement learning by introducing an attention-based method that projects inputs into an order-invariant representation space, reducing the search space by a factor of m! for m objects and demonstrating improved sample efficiency on various tasks.
Although deep reinforcement learning has advanced significantly over the past several years, sample efficiency remains a major challenge. Careful choice of input representations can help improve efficiency depending on the structure present in the problem. In this work, we present an attention-based method to project inputs into an efficient representation space that is invariant under changes to input ordering. We show that our proposed representation results in a search space that is a factor of m! smaller for inputs of m objects. Our experiments demonstrate improvements in sample efficiency for policy gradient methods on a variety of tasks. We show that our representation allows us to solve problems that are otherwise intractable when using naive approaches.