On Convergence of Epanechnikov Mean Shift
This work addresses a theoretical gap for a widely used clustering algorithm, making it more reliable for large and high-dimensional datasets, though it is incremental as it builds on existing methods.
The paper tackled the lack of theoretical convergence guarantees for Epanechnikov Mean Shift due to non-smooth kernel functions, showing it could terminate at non-critical points and proposing a modified version that guarantees convergence to local maxima within finite iterations, with experiments showing good performance compared to K-means and EM algorithms.
Epanechnikov Mean Shift is a simple yet empirically very effective algorithm for clustering. It localizes the centroids of data clusters via estimating modes of the probability distribution that generates the data points, using the `optimal' Epanechnikov kernel density estimator. However, since the procedure involves non-smooth kernel density functions, the convergence behavior of Epanechnikov mean shift lacks theoretical support as of this writing---most of the existing analyses are based on smooth functions and thus cannot be applied to Epanechnikov Mean Shift. In this work, we first show that the original Epanechnikov Mean Shift may indeed terminate at a non-critical point, due to the non-smoothness nature. Based on our analysis, we propose a simple remedy to fix it. The modified Epanechnikov Mean Shift is guaranteed to terminate at a local maximum of the estimated density, which corresponds to a cluster centroid, within a finite number of iterations. We also propose a way to avoid running the Mean Shift iterates from every data point, while maintaining good clustering accuracies under non-overlapping spherical Gaussian mixture models. This further pushes Epanechnikov Mean Shift to handle very large and high-dimensional data sets. Experiments show surprisingly good performance compared to the Lloyd's K-means algorithm and the EM algorithm.