Multimodal diffusion geometry by joint diagonalization of Laplacians
This work addresses multimodal data analysis for researchers in spectral methods, though it appears incremental as it builds on existing spectral clustering approaches.
The authors tackled the problem of extending diffusion geometry to multimodal data by jointly diagonalizing Laplacian matrices, resulting in improved structure capture for manifold learning, retrieval, and clustering tasks.
We construct an extension of diffusion geometry to multiple modalities through joint approximate diagonalization of Laplacian matrices. This naturally extends classical data analysis tools based on spectral geometry, such as diffusion maps and spectral clustering. We provide several synthetic and real examples of manifold learning, retrieval, and clustering demonstrating that the joint diffusion geometry frequently better captures the inherent structure of multi-modal data. We also show that many previous attempts to construct multimodal spectral clustering can be seen as particular cases of joint approximate diagonalization of the Laplacians.