CHEW: A Dataset of CHanging Events in Wikipedia
This work addresses the problem of evaluating LLMs' temporal reasoning for researchers, but it is incremental as it focuses on a specific dataset and task.
The authors introduced CHEW, a dataset of changing events in Wikipedia, and used it to probe LLMs for timeline understanding, finding that LLMs struggle to construct accurate timelines despite having temporal information available.
We introduce CHEW, a novel dataset of changing events in Wikipedia expressed in naturally occurring text. We use CHEW for probing LLMs for their timeline understanding of Wikipedia entities and events in generative and classification experiments. Our results suggest that LLMs, despite having temporal information available, struggle to construct accurate timelines. We further show the usefulness of CHEW-derived embeddings for identifying meaning shift.