CoVariance Filters and Neural Networks over Hilbert Spaces
This work addresses a foundational gap in machine learning for signals in infinite-dimensional spaces, with potential applications in domains like time-series analysis, but it is incremental as it builds on existing covariance network concepts.
The authors tackled the problem of extending covariance neural networks to infinite-dimensional Hilbert spaces by introducing Hilbert coVariance Filters and Networks, and they demonstrated robust performance in time-series classification tasks compared to MLP and FPCA-based classifiers.
CoVariance Neural Networks (VNNs) perform graph convolutions on the empirical covariance matrix of signals defined over finite-dimensional Hilbert spaces, motivated by robustness and transferability properties. Yet, little is known about how these arguments extend to infinite-dimensional Hilbert spaces. In this work, we take a first step by introducing a novel convolutional learning framework for signals defined over infinite-dimensional Hilbert spaces, centered on the (empirical) covariance operator. We constructively define Hilbert coVariance Filters (HVFs) and design Hilbert coVariance Networks (HVNs) as stacks of HVF filterbanks with nonlinear activations. We propose a principled discretization procedure, and we prove that empirical HVFs can recover the Functional PCA (FPCA) of the filtered signals. We then describe the versatility of our framework with examples ranging from multivariate real-valued functions to reproducing kernel Hilbert spaces. Finally, we validate HVNs on both synthetic and real-world time-series classification tasks, showing robust performance compared to MLP and FPCA-based classifiers.