Temporal Common Sense Acquisition with Minimal Supervision
This work addresses the problem of costly annotation and implicit information for temporal common sense in natural language processing, offering a model that improves performance on specific temporal tasks, though it is incremental as it builds on existing language model approaches.
The paper tackles the challenge of acquiring temporal common sense (e.g., event duration and frequency) from text by proposing TACOLM, a temporal common sense language model that uses explicit and implicit mentions from a large corpus. It shows quality predictions on datasets like UDST and RealNews, and produces better representations than BERT for tasks such as duration comparison and temporal QA on benchmarks like TimeBank and MCTACO.
Temporal common sense (e.g., duration and frequency of events) is crucial for understanding natural language. However, its acquisition is challenging, partly because such information is often not expressed explicitly in text, and human annotation on such concepts is costly. This work proposes a novel sequence modeling approach that exploits explicit and implicit mentions of temporal common sense, extracted from a large corpus, to build TACOLM, a temporal common sense language model. Our method is shown to give quality predictions of various dimensions of temporal common sense (on UDST and a newly collected dataset from RealNews). It also produces representations of events for relevant tasks such as duration comparison, parent-child relations, event coreference and temporal QA (on TimeBank, HiEVE and MCTACO) that are better than using the standard BERT. Thus, it will be an important component of temporal NLP.