What-if I ask you to explain: Explaining the effects of perturbations in procedural text
This addresses the problem of process comprehension in AI for domains like education or technical documentation, though it appears incremental as it builds on existing benchmarks and methods.
The paper tackles the task of explaining the effects of perturbations in procedural text, presenting QUARTET, a system that constructs such explanations and achieves better performance than baselines on a process comprehension benchmark, with a 7% absolute F1 improvement on a downstream QA task.
We address the task of explaining the effects of perturbations in procedural text, an important test of process comprehension. Consider a passage describing a rabbit's life-cycle: humans can easily explain the effect on the rabbit population if a female rabbit becomes ill -- i.e., the female rabbit would not become pregnant, and as a result not have babies leading to a decrease in rabbit population. We present QUARTET, a system that constructs such explanations from paragraphs, by modeling the explanation task as a multitask learning problem. QUARTET provides better explanations (based on the sentences in the procedural text) compared to several strong baselines on a recent process comprehension benchmark. We also present a surprising secondary effect: our model also achieves a new SOTA with a 7% absolute F1 improvement on a downstream QA task. This illustrates that good explanations do not have to come at the expense of end task performance.