Learnable Similarity and Dissimilarity Guided Symmetric Non-Negative Matrix Factorization
This work addresses a specific bottleneck in clustering algorithms for data analysis, offering an incremental improvement in efficiency and discriminative power.
The paper tackles the problem of constructing a reliable similarity matrix for symmetric nonnegative matrix factorization (SymNMF) in clustering, where traditional k-nearest neighbor methods can mislead due to unreliable neighbors, and proposes a learnable weighted k-NN graph with a dissimilarity matrix and orthogonality regularization, achieving improved clustering performance as demonstrated by comparisons with nine state-of-the-art methods on eight datasets.
Symmetric nonnegative matrix factorization (SymNMF) is a powerful tool for clustering, which typically uses the $k$-nearest neighbor ($k$-NN) method to construct similarity matrix. However, $k$-NN may mislead clustering since the neighbors may belong to different clusters, and its reliability generally decreases as $k$ grows. In this paper, we construct the similarity matrix as a weighted $k$-NN graph with learnable weight that reflects the reliability of each $k$-th NN. This approach reduces the search space of the similarity matrix learning to $n - 1$ dimension, as opposed to the $\mathcal{O}(n^2)$ dimension of existing methods, where $n$ represents the number of samples. Moreover, to obtain a discriminative similarity matrix, we introduce a dissimilarity matrix with a dual structure of the similarity matrix, and propose a new form of orthogonality regularization with discussions on its geometric interpretation and numerical stability. An efficient alternative optimization algorithm is designed to solve the proposed model, with theoretically guarantee that the variables converge to a stationary point that satisfies the KKT conditions. The advantage of the proposed model is demonstrated by the comparison with nine state-of-the-art clustering methods on eight datasets. The code is available at \url{https://github.com/lwl-learning/LSDGSymNMF}.