A Method of Selective Attention for Reservoir Based Agents
This addresses training efficiency for deep reinforcement learning practitioners, though it appears incremental as it builds on existing selective attention methods.
The paper tackles the problem of slow training in deep reinforcement learning agents due to irrelevant input dimensions by introducing a high-dimensional masking module for selective attention. The result is a four-fold training speedup compared to no input suppression and a two-fold speedup over layer normalization.
Training of deep reinforcement learning agents is slowed considerably by the presence of input dimensions that do not usefully condition the reward function. Existing modules such as layer normalization can be trained with weight decay to act as a form of selective attention, i.e. an input mask, that shrinks the scale of unnecessary inputs, which in turn accelerates training of the policy. However, we find a surprising result that adding numerous parameters to the computation of the input mask results in much faster training. A simple, high dimensional masking module is compared with layer normalization and a model without any input suppression. The high dimensional mask resulted in a four-fold speedup in training over the null hypothesis and a two-fold speedup in training over the layer normalization method.