SiT: Symmetry-Invariant Transformers for Generalisation in Reinforcement Learning
This addresses the problem of poor generalization in reinforcement learning for AI agents, though it appears incremental as it builds on existing transformer methods with symmetry enhancements.
The paper tackled the challenge of deploying trained reinforcement learning policies to new or semantically-similar environments by introducing SiT, a symmetry-invariant transformer that improved generalization, achieving superior performance over ViTs on MiniGrid and Procgen benchmarks and better sample efficiency on Atari 100k and CIFAR10.
An open challenge in reinforcement learning (RL) is the effective deployment of a trained policy to new or slightly different situations as well as semantically-similar environments. We introduce Symmetry-Invariant Transformer (SiT), a scalable vision transformer (ViT) that leverages both local and global data patterns in a self-supervised manner to improve generalisation. Central to our approach is Graph Symmetric Attention, which refines the traditional self-attention mechanism to preserve graph symmetries, resulting in invariant and equivariant latent representations. We showcase SiT's superior generalization over ViTs on MiniGrid and Procgen RL benchmarks, and its sample efficiency on Atari 100k and CIFAR10.