The Proof is in the Almond Cookies
This work addresses the challenge of processing how-to instructions for AI assistants to support human chefs, potentially aiding aging adults or people with impairments, but it is incremental as it builds on existing methods in a specific domain.
The paper tackles the problem of computational recipe understanding for AI cooking assistants by proposing a narrative-based approach that models recipes as rich structures using language processing, ontologies, and mental simulation, resulting in capabilities to handle recipe language challenges, optimize robot planning, measure AI task understanding, and enable language-independent annotations.
This paper presents a case study on how to process cooking recipes (and more generally, how-to instructions) in a way that makes it possible for a robot or artificial cooking assistant to support human chefs in the kitchen. Such AI assistants would be of great benefit to society, as they can help to sustain the autonomy of aging adults or people with a physical impairment, or they may reduce the stress in a professional kitchen. We propose a novel approach to computational recipe understanding that mimics the human sense-making process, which is narrative-based. Using an English recipe for almond crescent cookies as illustration, we show how recipes can be modelled as rich narrative structures by integrating various knowledge sources such as language processing, ontologies, and mental simulation. We show how such narrative structures can be used for (a) dealing with the challenges of recipe language, such as zero anaphora, (b) optimizing a robot's planning process, (c) measuring how well an AI system understands its current tasks, and (d) allowing recipe annotations to become language-independent.