Transformers as Policies for Variable Action Environments
This work addresses the challenge of efficient reinforcement learning in environments with variable actions, though it appears incremental as it adapts an existing architecture to a specific domain.
The paper tackled the problem of using transformer encoders as policies in variable action environments, achieving a higher return with half the computational resources compared to the next-best RL agent using GridNet in the Gym-μRTS environment.
In this project we demonstrate the effectiveness of the transformer encoder as a viable architecture for policies in variable action environments. Using it, we train an agent using Proximal Policy Optimisation (PPO) on multiple maps against scripted opponents in the Gym-$μ$RTS environment. The final agent is able to achieve a higher return using half the computational resources of the next-best RL agent, which used the GridNet architecture. The source code and pre-trained models are available here: https://github.com/NiklasZ/transformers-for-variable-action-envs