On Non-asymptotic Theory of Recurrent Neural Networks in Temporal Point Processes
This work provides theoretical foundations for neural TPPs, bridging the gap between practical applications and neural network theory, though it is incremental as it builds on existing RNN-TPP methods.
The paper tackles the lack of theoretical understanding of recurrent neural network-based temporal point processes (RNN-TPPs) by establishing excess risk bounds, showing that an RNN-TPP with up to four layers can achieve vanishing generalization errors.
Temporal point process (TPP) is an important tool for modeling and predicting irregularly timed events across various domains. Recently, the recurrent neural network (RNN)-based TPPs have shown practical advantages over traditional parametric TPP models. However, in the current literature, it remains nascent in understanding neural TPPs from theoretical viewpoints. In this paper, we establish the excess risk bounds of RNN-TPPs under many well-known TPP settings. We especially show that an RNN-TPP with no more than four layers can achieve vanishing generalization errors. Our technical contributions include the characterization of the complexity of the multi-layer RNN class, the construction of $\tanh$ neural networks for approximating dynamic event intensity functions, and the truncation technique for alleviating the issue of unbounded event sequences. Our results bridge the gap between TPP's application and neural network theory.