Deep Continuous-Time State-Space Models for Marked Event Sequences
This work addresses a fundamental challenge in modeling event sequences for applications like healthcare and finance, representing an incremental advance by combining deep state-space techniques with classical Hawkes processes.
The authors tackled the problem of modeling marked temporal point processes (MTPPs) for event sequences with irregular time intervals by proposing the state-space point process (S2P2) model, which achieved state-of-the-art predictive likelihoods with an average 33% improvement over existing methods across eight real-world datasets.
Marked temporal point processes (MTPPs) model sequences of events occurring at irregular time intervals, with wide-ranging applications in fields such as healthcare, finance and social networks. We propose the state-space point process (S2P2) model, a novel and performant model that leverages techniques derived for modern deep state-space models (SSMs) to overcome limitations of existing MTPP models, while simultaneously imbuing strong inductive biases for continuous-time event sequences that other discrete sequence models (i.e., RNNs, transformers) do not capture. Inspired by the classical linear Hawkes processes, we propose an architecture that interleaves stochastic jump differential equations with nonlinearities to create a highly expressive intensity-based MTPP model, without the need for restrictive parametric assumptions for the intensity. Our approach enables efficient training and inference with a parallel scan, bringing linear complexity and sublinear scaling while retaining expressivity to MTPPs. Empirically, S2P2 achieves state-of-the-art predictive likelihoods across eight real-world datasets, delivering an average improvement of 33% over the best existing approaches.