Alignment-based compositional semantics for instruction following
This addresses the challenge of instruction following for AI systems, but it appears incremental as it builds on existing alignment-based approaches.
The paper tackles the problem of interpreting natural language instructions by modeling them as a search over plans, scoring action sequences based on text and environment observations, and it outperforms strong baselines on multiple benchmark tasks, achieving new state-of-the-art results.
This paper describes an alignment-based model for interpreting natural language instructions in context. We approach instruction following as a search over plans, scoring sequences of actions conditioned on structured observations of text and the environment. By explicitly modeling both the low-level compositional structure of individual actions and the high-level structure of full plans, we are able to learn both grounded representations of sentence meaning and pragmatic constraints on interpretation. To demonstrate the model's flexibility, we apply it to a diverse set of benchmark tasks. On every task, we outperform strong task-specific baselines, and achieve several new state-of-the-art results.