Be Consistent! Improving Procedural Text Comprehension using Label Consistency
This work addresses the challenge of understanding dynamic changes in procedural texts, such as recipes or scientific processes, for applications in natural language processing, though it is incremental as it builds on existing methods with a consistency-based enhancement.
The paper tackles procedural text comprehension by tracking entity property changes over time, introducing a learning framework that enforces label consistency across multiple descriptions to improve prediction performance, achieving significant F1 gains over prior state-of-the-art on the ProPara benchmark.
Our goal is procedural text comprehension, namely tracking how the properties of entities (e.g., their location) change with time given a procedural text (e.g., a paragraph about photosynthesis, a recipe). This task is challenging as the world is changing throughout the text, and despite recent advances, current systems still struggle with this task. Our approach is to leverage the fact that, for many procedural texts, multiple independent descriptions are readily available, and that predictions from them should be consistent (label consistency). We present a new learning framework that leverages label consistency during training, allowing consistency bias to be built into the model. Evaluation on a standard benchmark dataset for procedural text, ProPara (Dalvi et al., 2018), shows that our approach significantly improves prediction performance (F1) over prior state-of-the-art systems.