Cell-Free Latent Go-Explore
This addresses the challenge of exploration in RL for scenarios where domain knowledge is unavailable, offering a more robust solution, though it builds incrementally on the Go-Explore paradigm.
The paper tackles the problem of exploration in reinforcement learning by introducing Latent Go-Explore (LGE), a method that generalizes Go-Explore without requiring domain knowledge or cell partitioning, and it outperforms state-of-the-art algorithms on hard-exploration environments like Montezuma's Revenge.
In this paper, we introduce Latent Go-Explore (LGE), a simple and general approach based on the Go-Explore paradigm for exploration in reinforcement learning (RL). Go-Explore was initially introduced with a strong domain knowledge constraint for partitioning the state space into cells. However, in most real-world scenarios, drawing domain knowledge from raw observations is complex and tedious. If the cell partitioning is not informative enough, Go-Explore can completely fail to explore the environment. We argue that the Go-Explore approach can be generalized to any environment without domain knowledge and without cells by exploiting a learned latent representation. Thus, we show that LGE can be flexibly combined with any strategy for learning a latent representation. Our results indicate that LGE, although simpler than Go-Explore, is more robust and outperforms state-of-the-art algorithms in terms of pure exploration on multiple hard-exploration environments including Montezuma's Revenge. The LGE implementation is available as open-source at https://github.com/qgallouedec/lge.