VALAN: Vision and Language Agent Navigation
This framework addresses the problem of building embodied agents for grounded language understanding tasks, but it is incremental as it builds on existing SEED RL architecture with minimal abstractions.
The paper introduces VALAN, a lightweight and scalable software framework based on SEED RL for developing and evaluating deep reinforcement learning agents in vision-and-language navigation tasks, such as instruction-conditioned indoor navigation in photo-realistic environments like Matterport3D and Google StreetView.
VALAN is a lightweight and scalable software framework for deep reinforcement learning based on the SEED RL architecture. The framework facilitates the development and evaluation of embodied agents for solving grounded language understanding tasks, such as Vision-and-Language Navigation and Vision-and-Dialog Navigation, in photo-realistic environments, such as Matterport3D and Google StreetView. We have added a minimal set of abstractions on top of SEED RL allowing us to generalize the architecture to solve a variety of other RL problems. In this article, we will describe VALAN's software abstraction and architecture, and also present an example of using VALAN to design agents for instruction-conditioned indoor navigation.