LiFT: Unsupervised Reinforcement Learning with Foundation Models as Teachers
This work addresses the challenge of unsupervised skill discovery in reinforcement learning for AI agents, though it appears incremental by building on existing foundation models.
The authors tackled the problem of enabling reinforcement learning agents to acquire semantically meaningful behaviors without human feedback by using foundation models as teachers, achieving successful skill learning in the challenging MineDojo environment where prior methods failed.
We propose a framework that leverages foundation models as teachers, guiding a reinforcement learning agent to acquire semantically meaningful behavior without human feedback. In our framework, the agent receives task instructions grounded in a training environment from large language models. Then, a vision-language model guides the agent in learning the multi-task language-conditioned policy by providing reward feedback. We demonstrate that our method can learn semantically meaningful skills in a challenging open-ended MineDojo environment while prior unsupervised skill discovery methods struggle. Additionally, we discuss observed challenges of using off-the-shelf foundation models as teachers and our efforts to address them.