Scene Memory Transformer for Embodied Agents in Long-Horizon Tasks
This addresses the challenge for robotic agents that need to rely on long-term history for decision-making in complex, partially observable settings.
The paper tackles the problem of enabling embodied agents to perform long-horizon tasks in partially observable environments by proposing the Scene Memory Transformer (SMT), which embeds observations into memory and uses attention to exploit spatio-temporal dependencies, resulting in superior performance on visual navigation tasks compared to existing policies.
Many robotic applications require the agent to perform long-horizon tasks in partially observable environments. In such applications, decision making at any step can depend on observations received far in the past. Hence, being able to properly memorize and utilize the long-term history is crucial. In this work, we propose a novel memory-based policy, named Scene Memory Transformer (SMT). The proposed policy embeds and adds each observation to a memory and uses the attention mechanism to exploit spatio-temporal dependencies. This model is generic and can be efficiently trained with reinforcement learning over long episodes. On a range of visual navigation tasks, SMT demonstrates superior performance to existing reactive and memory-based policies by a margin.