MaestroMotif: Skill Design from Artificial Intelligence Feedback
This addresses the challenge of making AI skill design more accessible and effective for developers and researchers, though it appears incremental as it builds on existing LLM and reinforcement learning techniques.
The paper tackles the problem of designing AI skills from natural language descriptions by introducing MaestroMotif, a method that uses LLM feedback and reinforcement learning to create and reuse skills, resulting in agents that outperform existing approaches in complex tasks like NetHack.
Describing skills in natural language has the potential to provide an accessible way to inject human knowledge about decision-making into an AI system. We present MaestroMotif, a method for AI-assisted skill design, which yields high-performing and adaptable agents. MaestroMotif leverages the capabilities of Large Language Models (LLMs) to effectively create and reuse skills. It first uses an LLM's feedback to automatically design rewards corresponding to each skill, starting from their natural language description. Then, it employs an LLM's code generation abilities, together with reinforcement learning, for training the skills and combining them to implement complex behaviors specified in language. We evaluate MaestroMotif using a suite of complex tasks in the NetHack Learning Environment (NLE), demonstrating that it surpasses existing approaches in both performance and usability.