MinePlanner: A Benchmark for Long-Horizon Planning in Large Minecraft Worlds
This provides a domain-specific benchmark for evaluating planning systems in complex environments, though it is incremental as it builds on existing planning frameworks.
The authors introduced MinePlanner, a new benchmark for long-horizon planning tasks in Minecraft, containing 45 tasks and tools for automatic instance generation, and found that state-of-the-art planners struggle with challenges like scaling to thousands of objects.
We propose a new benchmark for planning tasks based on the Minecraft game. Our benchmark contains 45 tasks overall, but also provides support for creating both propositional and numeric instances of new Minecraft tasks automatically. We benchmark numeric and propositional planning systems on these tasks, with results demonstrating that state-of-the-art planners are currently incapable of dealing with many of the challenges advanced by our new benchmark, such as scaling to instances with thousands of objects. Based on these results, we identify areas of improvement for future planners. Our framework is made available at https://github.com/IretonLiu/mine-pddl/.