K-means for Evolving Data Streams
This work is significant for researchers and practitioners dealing with continuous, high-volume unsupervised data streams, particularly in scenarios prone to concept drift, by offering a method to maintain clustering accuracy without explicit drift detection.
The paper addresses the challenge of clustering evolving data streams where concept drift can occur. It proposes a surrogate error function for Streaming K-means that does not require explicit concept drift detection, and an algorithm to minimize this error with each new data batch. Experiments show improved converged error with non-trivial initialization methods.
Currently the amount of data produced worldwide is increasing beyond measure, thus a high volume of unsupervised data must be processed continuously. One of the main unsupervised data analysis is clustering. In streaming data scenarios, the data is composed by an increasing sequence of batches of samples where the concept drift phenomenon may happen. In this paper, we formally define the Streaming $K$-means(S$K$M) problem, which implies a restart of the error function when a concept drift occurs. We propose a surrogate error function that does not rely on concept drift detection. We proof that the surrogate is a good approximation of the S$K$M error. Hence, we suggest an algorithm which minimizes this alternative error each time a new batch arrives. We present some initialization techniques for streaming data scenarios as well. Besides providing theoretical results, experiments demonstrate an improvement of the converged error for the non-trivial initialization methods.