Collecting Interactive Multi-modal Datasets for Grounded Language Understanding
This work addresses the challenge of building machines with human-like adaptive learning capabilities, which is incremental as it focuses on dataset creation rather than novel algorithmic breakthroughs.
The paper tackled the problem of enabling machines to adapt to new tasks through imitation or natural language instructions by formalizing a collaborative embodied agent task, developing a scalable data collection tool, and collecting the first dataset for interactive grounded language understanding.
Human intelligence can remarkably adapt quickly to new tasks and environments. Starting from a very young age, humans acquire new skills and learn how to solve new tasks either by imitating the behavior of others or by following provided natural language instructions. To facilitate research which can enable similar capabilities in machines, we made the following contributions (1) formalized the collaborative embodied agent using natural language task; (2) developed a tool for extensive and scalable data collection; and (3) collected the first dataset for interactive grounded language understanding.