Tracking entities in technical procedures -- a new dataset and baselines
This addresses the problem of understanding procedures from temporal texts for researchers, but it is incremental as it primarily provides a new dataset and baselines.
The authors introduced TechTrack, a dataset of 1351 technical procedures with over 1200 unique entities, and found that state-of-the-art models perform significantly below human levels on entity tracking.
We introduce TechTrack, a new dataset for tracking entities in technical procedures. The dataset, prepared by annotating open domain articles from WikiHow, consists of 1351 procedures, e.g., "How to connect a printer", identifies more than 1200 unique entities with an average of 4.7 entities per procedure. We evaluate the performance of state-of-the-art models on the entity-tracking task and find that they are well below the human annotation performance. We describe how TechTrack can be used to take forward the research on understanding procedures from temporal texts.