Combining LLMs with Logic-Based Framework to Explain MCTS
This addresses the problem of interpretability in complex AI planning for users needing trustworthy explanations, though it is incremental as it builds on existing methods for explanation.
The paper tackles the lack of trust in AI for sequential planning by designing a framework that combines LLMs with Computational Tree Logic to provide natural language explanations for the MCTS algorithm, demonstrating strong performance in accuracy and factual consistency.
In response to the lack of trust in Artificial Intelligence (AI) for sequential planning, we design a Computational Tree Logic-guided large language model (LLM)-based natural language explanation framework designed for the Monte Carlo Tree Search (MCTS) algorithm. MCTS is often considered challenging to interpret due to the complexity of its search trees, but our framework is flexible enough to handle a wide range of free-form post-hoc queries and knowledge-based inquiries centered around MCTS and the Markov Decision Process (MDP) of the application domain. By transforming user queries into logic and variable statements, our framework ensures that the evidence obtained from the search tree remains factually consistent with the underlying environmental dynamics and any constraints in the actual stochastic control process. We evaluate the framework rigorously through quantitative assessments, where it demonstrates strong performance in terms of accuracy and factual consistency.