Clustering by Deep Nearest Neighbor Descent (D-NND): A Density-based Parameter-Insensitive Clustering Method
This addresses the problem of parameter tuning in density estimation for clustering, offering a more robust method for data analysis, though it appears incremental as it builds on existing density-based approaches.
The paper tackles the challenge of parameter sensitivity in density-based clustering by introducing Deep Nearest Neighbor Descent (D-NND), a hierarchical method that learns density structures layer by layer to avoid over-smoothing and reduce ripple noise, resulting in strong cluster discovery and remarkable reliability with insensitivity to parameters.
Most density-based clustering methods largely rely on how well the underlying density is estimated. However, density estimation itself is also a challenging problem, especially the determination of the kernel bandwidth. A large bandwidth could lead to the over-smoothed density estimation in which the number of density peaks could be less than the true clusters, while a small bandwidth could lead to the under-smoothed density estimation in which spurious density peaks, or called the "ripple noise", would be generated in the estimated density. In this paper, we propose a density-based hierarchical clustering method, called the Deep Nearest Neighbor Descent (D-NND), which could learn the underlying density structure layer by layer and capture the cluster structure at the same time. The over-smoothed density estimation could be largely avoided and the negative effect of the under-estimated cases could be also largely reduced. Overall, D-NND presents not only the strong capability of discovering the underlying cluster structure but also the remarkable reliability due to its insensitivity to parameters.