Enabling DBSCAN for Very Large-Scale High-Dimensional Spaces
This addresses a scalability problem for users of DBSCAN in big data applications, though it is incremental as it builds on existing DBSCAN with a compression technique.
The paper tackles the computational infeasibility of DBSCAN for large-scale, high-dimensional datasets by proposing a spectral data compression method, which improves solution quality and enables more accurate results.
DBSCAN is one of the most important non-parametric unsupervised data analysis tools. By applying DBSCAN to a dataset, two key analytical results can be obtained: (1) clustering data points based on density distribution and (2) identifying outliers in the dataset. However, the time complexity of the DBSCAN algorithm is $O(n^2 β)$, where $n$ is the number of data points and $β= O(D)$, with $D$ representing the dimensionality of the data space. As a result, DBSCAN becomes computationally infeasible when both $n$ and $D$ are large. In this paper, we propose a DBSCAN method based on spectral data compression, capable of efficiently processing datasets with a large number of data points ($n$) and high dimensionality ($D$). By preserving only the most critical structural information during the compression process, our method effectively removes substantial redundancy and noise. Consequently, the solution quality of DBSCAN is significantly improved, enabling more accurate and reliable results.