Natural Language Reinforcement Learning
This work addresses the problem of enabling deeper understanding and more active learning in AI agents for researchers and practitioners in reinforcement learning, though it appears incremental as an extension of RL principles.
The paper tackles the limitation of traditional Reinforcement Learning (RL) by introducing Natural Language Reinforcement Learning (NLRL), which redefines value as an interpretable linguistic narrative, and demonstrates its effectiveness and efficiency across 4 multi-step agentic tasks.
Artificial intelligence progresses towards the "Era of Experience," where agents are expected to learn from continuous, grounded interaction. We argue that traditional Reinforcement Learning (RL), which typically represents value as a scalar, can restrict agent's deep understanding of environments and hinders the active, deliberative learning crucial for navigating this new paradigm. To address the issue, we introduce Natural Language Reinforcement Learning (NLRL), a framework that extends RL principles into natural language counterparts. Central to NLRL is the Language Value Function (LVF), which redefines value as an interpretable linguistic narrative articulating the rationale behind an evaluation. NLRL further extends this concept to core RL components, including policy, the Bellman equation, and policy iteration. Leveraging recent advancements in Large Language Models (LLMs), NLRL can be practically implemented to achieve RL-like policy and value training through unsupervised environment interactions. Experiments over 4 multi-step agentic tasks demonstrate NLRL's effectiveness, efficiency, and its potential to foster deeper understanding and more active learning strategies.