Time-Parameterized Convolutional Neural Networks for Irregularly Sampled Time Series
This addresses a challenge in domains with irregular time series data, such as healthcare or finance, by enabling more efficient and interpretable convolutional approaches, though it is incremental as it adapts existing CNN concepts to a new setting.
The paper tackles the problem of modeling irregularly sampled multivariate time series, where standard neural networks like CNNs and RNNs struggle due to regular spacing assumptions, by proposing a time-parameterized convolutional neural network (TPCNN) that uses time-explicitly initialized kernels; experimental results show competitive performance and significantly higher efficiency compared to state-of-the-art methods on interpolation and classification tasks.
Irregularly sampled multivariate time series are ubiquitous in several application domains, leading to sparse, not fully-observed and non-aligned observations across different variables. Standard sequential neural network architectures, such as recurrent neural networks (RNNs) and convolutional neural networks (CNNs), consider regular spacing between observation times, posing significant challenges to irregular time series modeling. While most of the proposed architectures incorporate RNN variants to handle irregular time intervals, convolutional neural networks have not been adequately studied in the irregular sampling setting. In this paper, we parameterize convolutional layers by employing time-explicitly initialized kernels. Such general functions of time enhance the learning process of continuous-time hidden dynamics and can be efficiently incorporated into convolutional kernel weights. We, thus, propose the time-parameterized convolutional neural network (TPCNN), which shares similar properties with vanilla convolutions but is carefully designed for irregularly sampled time series. We evaluate TPCNN on both interpolation and classification tasks involving real-world irregularly sampled multivariate time series datasets. Our experimental results indicate the competitive performance of the proposed TPCNN model which is also significantly more efficient than other state-of-the-art methods. At the same time, the proposed architecture allows the interpretability of the input series by leveraging the combination of learnable time functions that improve the network performance in subsequent tasks and expedite the inaugural application of convolutions in this field.