In-Context Learning of Temporal Point Processes with Foundation Inference Models
This work addresses the inefficiency of training separate models for each event sequence system in temporal point process modeling, offering a more scalable solution for researchers and practitioners in fields like healthcare or finance.
The authors tackled the problem of modeling event sequences with marked temporal point processes by pretraining a deep neural network to infer conditional intensity functions in-context, eliminating the need for separate specialized models per system. The result shows that this amortized approach matches the performance of specialized models on next-event prediction across benchmark datasets.
Modeling event sequences of multiple event types with marked temporal point processes (MTPPs) provides a principled way to uncover governing dynamical rules and predict future events. Current neural network approaches to MTPP inference rely on training separate, specialized models for each target system. We pursue a radically different approach: drawing on amortized inference and in-context learning, we pretrain a deep neural network to infer, in-context, the conditional intensity functions of event histories from a context defined by sets of event sequences. Pretraining is performed on a large synthetic dataset of MTPPs sampled from a broad distribution of Hawkes processes. Once pretrained, our Foundation Inference Model for Point Processes (FIM-PP) can estimate MTPPs from real-world data without any additional training, or be rapidly finetuned to target systems. Experiments show that this amortized approach matches the performance of specialized models on next-event prediction across common benchmark datasets. Our pretrained model, repository and tutorials will soon be available online