Optimized Monte Carlo Tree Search for Enhanced Decision Making in the FrozenLake Environment
This is an incremental improvement for reinforcement learning practitioners working on stochastic grid-world environments.
The paper tackles decision-making in the FrozenLake reinforcement learning environment by optimizing Monte Carlo Tree Search (MCTS) with cumulative reward and visit count tables and the UCT formula, resulting in improved reward maximization, success rates, and convergence time compared to baseline methods like MCTS with Policy and Q-Learning.
Monte Carlo Tree Search (MCTS) is a powerful algorithm for solving complex decision-making problems. This paper presents an optimized MCTS implementation applied to the FrozenLake environment, a classic reinforcement learning task characterized by stochastic transitions. The optimization leverages cumulative reward and visit count tables along with the Upper Confidence Bound for Trees (UCT) formula, resulting in efficient learning in a slippery grid world. We benchmark our implementation against other decision-making algorithms, including MCTS with Policy and Q-Learning, and perform a detailed comparison of their performance. The results demonstrate that our optimized approach effectively maximizes rewards and success rates while minimizing convergence time, outperforming baseline methods, especially in environments with inherent randomness.