Spatiotemporal Covariance Neural Networks
This addresses the problem of unstable covariance eigenvectors in PCA for researchers and practitioners dealing with multivariate time series in streaming and non-stationary environments, representing an incremental improvement over existing methods.
The paper tackled the instability of PCA-based methods in modeling spatiotemporal interactions for multivariate time series, particularly in dynamic and streaming settings, by introducing the SpatioTemporal coVariance Neural Network (STVNN), which proved to be orders of magnitude more stable than online temporal PCA and competitive in processing.
Modeling spatiotemporal interactions in multivariate time series is key to their effective processing, but challenging because of their irregular and often unknown structure. Statistical properties of the data provide useful biases to model interdependencies and are leveraged by correlation and covariance-based networks as well as by processing pipelines relying on principal component analysis (PCA). However, PCA and its temporal extensions suffer instabilities in the covariance eigenvectors when the corresponding eigenvalues are close to each other, making their application to dynamic and streaming data settings challenging. To address these issues, we exploit the analogy between PCA and graph convolutional filters to introduce the SpatioTemporal coVariance Neural Network (STVNN), a relational learning model that operates on the sample covariance matrix of the time series and leverages joint spatiotemporal convolutions to model the data. To account for the streaming and non-stationary setting, we consider an online update of the parameters and sample covariance matrix. We prove the STVNN is stable to the uncertainties introduced by these online estimations, thus improving over temporal PCA-based methods. Experimental results corroborate our theoretical findings and show that STVNN is competitive for multivariate time series processing, it adapts to changes in the data distribution, and it is orders of magnitude more stable than online temporal PCA.