GRILLBot: An Assistant for Real-World Tasks with Neural Semantic Parsing and Graph-Based Representations
It addresses the problem of assisting users in long-running, complex tasks through a multimodal assistant, representing an incremental advance in task-oriented dialogue systems.
The paper tackles the challenge of building a voice assistant for complex real-world tasks like cooking and home improvement, resulting in GRILLBot, which won the 2022 Alexa Prize TaskBot Challenge by enabling flexible task guidance with multimodal support.
GRILLBot is the winning system in the 2022 Alexa Prize TaskBot Challenge, moving towards the next generation of multimodal task assistants. It is a voice assistant to guide users through complex real-world tasks in the domains of cooking and home improvement. These are long-running and complex tasks that require flexible adjustment and adaptation. The demo highlights the core aspects, including a novel Neural Decision Parser for contextualized semantic parsing, a new "TaskGraph" state representation that supports conditional execution, knowledge-grounded chit-chat, and automatic enrichment of tasks with images and videos.