Reinforcement Learning Problem Solving with Large Language Models
This work addresses the untapped potential of conversational RL problem-solving for domains like research and legal workflows, but it appears incremental as it builds on existing LLM and RL paradigms.
The study tackles the problem of using Large Language Models (LLMs) as Reinforcement Learning (RL) agents by formulating RL problems as prompting tasks, demonstrating iterative prompting for policy learning and optimization, and applying it to case studies like 'Research Scientist' and 'Legal Matter Intake' workflows.
Large Language Models (LLMs) encapsulate an extensive amount of world knowledge, and this has enabled their application in various domains to improve the performance of a variety of Natural Language Processing (NLP) tasks. This has also facilitated a more accessible paradigm of conversation-based interactions between humans and AI systems to solve intended problems. However, one interesting avenue that shows untapped potential is the use of LLMs as Reinforcement Learning (RL) agents to enable conversational RL problem solving. Therefore, in this study, we explore the concept of formulating Markov Decision Process-based RL problems as LLM prompting tasks. We demonstrate how LLMs can be iteratively prompted to learn and optimize policies for specific RL tasks. In addition, we leverage the introduced prompting technique for episode simulation and Q-Learning, facilitated by LLMs. We then show the practicality of our approach through two detailed case studies for "Research Scientist" and "Legal Matter Intake" workflows.