An Empirical Comparison of Neural Architectures for Reinforcement Learning in Partially Observable Environments
This work addresses performance optimization for reinforcement learning agents in partially observable settings, but it is incremental as it focuses on empirical comparisons of existing and evolved architectures.
The paper tackled reinforcement learning in partially observable environments by comparing three recurrent neural network architectures (LSTM, GRU, and evolved MUT1) and a variant using Advantage values, finding that GRU performed significantly better with less training episodes and CPU time, and Advantage learning tended to improve results.
This paper explores the performance of fitted neural Q iteration for reinforcement learning in several partially observable environments, using three recurrent neural network architectures: Long Short-Term Memory, Gated Recurrent Unit and MUT1, a recurrent neural architecture evolved from a pool of several thousands candidate architectures. A variant of fitted Q iteration, based on Advantage values instead of Q values, is also explored. The results show that GRU performs significantly better than LSTM and MUT1 for most of the problems considered, requiring less training episodes and less CPU time before learning a very good policy. Advantage learning also tends to produce better results.