Towards Benchmarking and Improving the Temporal Reasoning Capability of Large Language Models
This work addresses the need for better temporal reasoning in AI systems, which is crucial for applications like question answering, but it is incremental as it builds on existing datasets and methods.
The paper tackles the problem of evaluating and improving temporal reasoning in large language models by introducing a comprehensive dataset (tempreason) and a novel learning framework, achieving demonstrated effectiveness across multiple QA settings.
Reasoning about time is of fundamental importance. Many facts are time-dependent. For example, athletes change teams from time to time, and different government officials are elected periodically. Previous time-dependent question answering (QA) datasets tend to be biased in either their coverage of time spans or question types. In this paper, we introduce a comprehensive probing dataset \tempreason to evaluate the temporal reasoning capability of large language models. Our dataset includes questions of three temporal reasoning levels. In addition, we also propose a novel learning framework to improve the temporal reasoning capability of large language models, based on temporal span extraction and time-sensitive reinforcement learning. We conducted experiments in closed book QA, open book QA, and reasoning QA settings and demonstrated the effectiveness of our approach. Our code and data are released on https://github.com/DAMO-NLP-SG/TempReason.