Temporal Shift Reinforcement Learning
This addresses a limitation in sequential decision-making for domains like robotics, but it is incremental as it builds on existing DRL methods.
The paper tackled the problem of traditional image-based Deep Reinforcement Learning lacking temporal learning by proposing Temporal Shift Reinforcement Learning (TSRL), which jointly learns temporal and spatial components without extra parameters, resulting in outperforming frame stacking on two Atari environments and beating state-of-the-art on one.
The function approximators employed by traditional image-based Deep Reinforcement Learning (DRL) algorithms usually lack a temporal learning component and instead focus on learning the spatial component. We propose a technique, Temporal Shift Reinforcement Learning (TSRL), wherein both temporal, as well as spatial components are jointly learned. Moreover, TSRL does not require additional parameters to perform temporal learning. We show that TSRL outperforms the commonly used frame stacking heuristic on both of the Atari environments we test on while beating the SOTA for one of them. This investigation has implications in the robotics as well as sequential decision-making domains.