Maximum Likelihood Estimation for Single Linkage Hierarchical Clustering
This work addresses uncertainty in hierarchical clustering for data analysis, but it appears incremental as it builds on existing SLHC methods with a new statistical model.
The authors tackled the problem of estimating dendrogram structures from single linkage hierarchical clustering under noisy conditions, introducing an approximate maximum likelihood estimator that outperformed standard SLHC in small-scale simulations.
We derive a statistical model for estimation of a dendrogram from single linkage hierarchical clustering (SLHC) that takes account of uncertainty through noise or corruption in the measurements of separation of data. Our focus is on just the estimation of the hierarchy of partitions afforded by the dendrogram, rather than the heights in the latter. The concept of estimating this "dendrogram structure'' is introduced, and an approximate maximum likelihood estimator (MLE) for the dendrogram structure is described. These ideas are illustrated by a simple Monte Carlo simulation that, at least for small data sets, suggests the method outperforms SLHC in the presence of noise.