SkillTree: Explainable Skill-Based Deep Reinforcement Learning for Long-Horizon Control Tasks
This addresses the problem of explainability in DRL for complex control tasks, offering skill-level transparency, though it is incremental as it builds on existing skill-based methods.
The paper tackles the lack of transparency in deep reinforcement learning for long-horizon control tasks by proposing SkillTree, a framework that reduces continuous action spaces to discrete skill spaces using a hierarchical approach with a differentiable decision tree, achieving performance comparable to neural networks in robotic arm control.
Deep reinforcement learning (DRL) has achieved remarkable success in various research domains. However, its reliance on neural networks results in a lack of transparency, which limits its practical applications. To achieve explainability, decision trees have emerged as a popular and promising alternative to neural networks. Nonetheless, due to their limited expressiveness, traditional decision trees struggle with high-dimensional long-horizon continuous control tasks. In this paper, we proposes SkillTree, a novel framework that reduces complex continuous action spaces into discrete skill spaces. Our hierarchical approach integrates a differentiable decision tree within the high-level policy to generate skill embeddings, which subsequently guide the low-level policy in executing skills. By making skill decisions explainable, we achieve skill-level explainability, enhancing the understanding of the decision-making process in complex tasks. Experimental results demonstrate that our method achieves performance comparable to skill-based neural networks in complex robotic arm control domains. Furthermore, SkillTree offers explanations at the skill level, thereby increasing the transparency of the decision-making process.