Language-guided Skill Learning with Temporal Variational Inference
This work addresses the problem of improving learning efficiency in long-horizon tasks for AI agents in simulated environments, representing an incremental advancement in skill learning methods.
The paper tackles skill discovery from expert demonstrations by using LLMs for initial trajectory segmentation and a hierarchical variational inference framework to merge segments into reusable skills, with an auxiliary objective based on Minimum Description Length to balance compression and reusability. The result shows that agents with this method accelerate learning and outperform baseline skill learning approaches on new long-horizon tasks in BabyAI and ALFRED environments.
We present an algorithm for skill discovery from expert demonstrations. The algorithm first utilizes Large Language Models (LLMs) to propose an initial segmentation of the trajectories. Following that, a hierarchical variational inference framework incorporates the LLM-generated segmentation information to discover reusable skills by merging trajectory segments. To further control the trade-off between compression and reusability, we introduce a novel auxiliary objective based on the Minimum Description Length principle that helps guide this skill discovery process. Our results demonstrate that agents equipped with our method are able to discover skills that help accelerate learning and outperform baseline skill learning approaches on new long-horizon tasks in BabyAI, a grid world navigation environment, as well as ALFRED, a household simulation environment.