EgoMap: Projective mapping and structured egocentric memory for Deep RL
This work addresses tasks with multi-step objectives in 3D environments for deep reinforcement learning, offering a novel method to improve performance without expert trajectories.
The paper tackled the challenge of localization, memorization, and planning in partially observable 3D environments for deep reinforcement learning by introducing EgoMap, a spatially structured neural memory architecture that outperforms standard recurrent and state-of-the-art agents with structured memory.
Tasks involving localization, memorization and planning in partially observable 3D environments are an ongoing challenge in Deep Reinforcement Learning. We present EgoMap, a spatially structured neural memory architecture. EgoMap augments a deep reinforcement learning agent's performance in 3D environments on challenging tasks with multi-step objectives. The EgoMap architecture incorporates several inductive biases including a differentiable inverse projection of CNN feature vectors onto a top-down spatially structured map. The map is updated with ego-motion measurements through a differentiable affine transform. We show this architecture outperforms both standard recurrent agents and state of the art agents with structured memory. We demonstrate that incorporating these inductive biases into an agent's architecture allows for stable training with reward alone, circumventing the expense of acquiring and labelling expert trajectories. A detailed ablation study demonstrates the impact of key aspects of the architecture and through extensive qualitative analysis, we show how the agent exploits its structured internal memory to achieve higher performance.