Visual Hindsight Self-Imitation Learning for Interactive Navigation
This work addresses the problem of inefficient learning in complex visual navigation for AI agents, offering a novel method that reduces reliance on dense rewards or expert data, though it is incremental in improving upon existing self-imitation techniques.
The paper tackles the challenge of sample inefficiency in interactive visual navigation tasks by proposing Visual Hindsight Self-Imitation Learning (VHS), which uses hindsight goal re-labeling and self-imitation to enhance learning from rare successful experiences, resulting in superior performance and sample efficiency compared to existing methods.
Interactive visual navigation tasks, which involve following instructions to reach and interact with specific targets, are challenging not only because successful experiences are very rare but also because the complex visual inputs require a substantial number of samples. Previous methods for these tasks often rely on intricately designed dense rewards or the use of expensive expert data for imitation learning. To tackle these challenges, we propose a novel approach, Visual Hindsight Self-Imitation Learning (VHS) for enhancing sample efficiency through hindsight goal re-labeling and self-imitation. We also introduce a prototypical goal embedding method derived from experienced goal observations, that is particularly effective in vision-based and partially observable environments. This embedding technique allows the agent to visually reinterpret its unsuccessful attempts, enabling vision-based goal re-labeling and self-imitation from enhanced successful experiences. Experimental results show that VHS outperforms existing techniques in interactive visual navigation tasks, confirming its superior performance and sample efficiency.