Distilling Event Sequence Knowledge From Large Language Models
This work addresses the challenge of noisy and incomplete event sequence data for researchers and practitioners in event analysis and prediction, representing an incremental improvement by hybridizing LLMs with Knowledge Graphs.
The paper tackles the problem of building probabilistic event models when clean structured event sequence data is unavailable by using Large Language Models (LLMs) to generate high-quality event sequences, guided by a Knowledge Graph with partial causal relations, and shows that this approach can fill knowledge gaps and enable discovery of more complex structured knowledge.
Event sequence models have been found to be highly effective in the analysis and prediction of events. Building such models requires availability of abundant high-quality event sequence data. In certain applications, however, clean structured event sequences are not available, and automated sequence extraction results in data that is too noisy and incomplete. In this work, we explore the use of Large Language Models (LLMs) to generate event sequences that can effectively be used for probabilistic event model construction. This can be viewed as a mechanism of distilling event sequence knowledge from LLMs. Our approach relies on a Knowledge Graph (KG) of event concepts with partial causal relations to guide the generative language model for causal event sequence generation. We show that our approach can generate high-quality event sequences, filling a knowledge gap in the input KG. Furthermore, we explore how the generated sequences can be leveraged to discover useful and more complex structured knowledge from pattern mining and probabilistic event models. We release our sequence generation code and evaluation framework, as well as corpus of event sequence data.