Spectral Discovery of Jointly Smooth Features for Multimodal Data
This method addresses the challenge of aligning multimodal data from different sensors, which is incremental as it builds on spectral techniques but offers specific improvements for applications like sleep analysis.
The authors tackled the problem of registering measurements from heterogeneous sensors by proposing a spectral method to derive functions that are jointly smooth across multiple manifolds, which achieved superior results in sleep stage identification compared to nonlinear CCA variants.
In this paper, we propose a spectral method for deriving functions that are jointly smooth on multiple observed manifolds. This allows us to register measurements of the same phenomenon by heterogeneous sensors, and to reject sensor-specific noise. Our method is unsupervised and primarily consists of two steps. First, using kernels, we obtain a subspace spanning smooth functions on each separate manifold. Then, we apply a spectral method to the obtained subspaces and discover functions that are jointly smooth on all manifolds. We show analytically that our method is guaranteed to provide a set of orthogonal functions that are as jointly smooth as possible, ordered by increasing Dirichlet energy from the smoothest to the least smooth. In addition, we show that the extracted functions can be efficiently extended to unseen data using the Nyström method. We demonstrate the proposed method on both simulated and real measured data and compare the results to nonlinear variants of the seminal Canonical Correlation Analysis (CCA). Particularly, we show superior results for sleep stage identification. In addition, we show how the proposed method can be leveraged for finding minimal realizations of parameter spaces of nonlinear dynamical systems.