Closed-Loop Verbal Reinforcement Learning for Task-Level Robotic Planning
This work addresses the need for transparent and adaptable robotic planning, though it appears incremental as it builds on existing methods like Behavior Trees and large language models.
The authors tackled the problem of interpretable task-level planning for mobile robots under execution uncertainty by proposing a Verbal Reinforcement Learning framework, which achieved explainable policy improvements and reliable deployment on a real robot performing multi-stage manipulation and navigation tasks.
We propose a new Verbal Reinforcement Learning (VRL) framework for interpretable task-level planning in mobile robotic systems operating under execution uncertainty. The framework follows a closed-loop architecture that enables iterative policy improvement through interaction with the physical environment. In our framework, executable Behavior Trees are repeatedly refined by a Large Language Model actor using structured natural-language feedback produced by a Vision-Language Model critic that observes the physical robot and execution traces. Unlike conventional reinforcement learning, policy updates in VRL occur directly at the symbolic planning level, without gradient-based optimization. This enables transparent reasoning, explicit causal feedback, and human-interpretable policy evolution. We validate the proposed framework on a real mobile robot performing a multi-stage manipulation and navigation task under execution uncertainty. Experimental results show that the framework supports explainable policy improvements, closed-loop adaptation to execution failures, and reliable deployment on physical robotic systems.