Generalizable Spectral Embedding with an Application to UMAP
This work improves SE for researchers and practitioners in dimensionality reduction, though it is incremental as it builds on SpectralNet.
The paper tackles the limitations of Spectral Embedding (SE) in generalizability, scalability, and eigenvectors separation by introducing Sep-SpectralNet, which extends SpectralNet with an efficient post-processing step to address all three issues, enabling generalizable UMAP visualization.
Spectral Embedding (SE) is a popular method for dimensionality reduction, applicable across diverse domains. Nevertheless, its current implementations face three prominent drawbacks which curtail its broader applicability: generalizability (i.e., out-of-sample extension), scalability, and eigenvectors separation. Existing SE implementations often address two of these drawbacks; however, they fall short in addressing the remaining one. In this paper, we introduce Sep-SpectralNet (eigenvector-separated SpectralNet), a SE implementation designed to address all three limitations. Sep-SpectralNet extends SpectralNet with an efficient post-processing step to achieve eigenvectors separation, while ensuring both generalizability and scalability. This method expands the applicability of SE to a wider range of tasks and can enhance its performance in existing applications. We empirically demonstrate Sep-SpectralNet's ability to consistently approximate and generalize SE, while maintaining SpectralNet's scalability. Additionally, we show how Sep-SpectralNet can be leveraged to enable generalizable UMAP visualization. Our codes are publicly available.