A Simple and Efficient Method to Compute a Single Linkage Dendrogram
This is an incremental improvement for researchers and practitioners in clustering and data analysis, offering a more memory-efficient approach to dendrogram computation.
The paper tackles the problem of computing a single linkage dendrogram efficiently by leveraging Prim's algorithm for MST construction to avoid storing the full distance matrix, and it introduces a method to recursively split the MST without extra computational cost.
We address the problem of computing a single linkage dendrogram. A possible approach is to: (i) Form an edge weighted graph $G$ over the data, with edge weights reflecting dissimilarities. (ii) Calculate the MST $T$ of $G$. (iii) Break the longest edge of $T$ thereby splitting it into subtrees $T_L$, $T_R$. (iv) Apply the splitting process recursively to the subtrees. This approach has the attractive feature that Prim's algorithm for MST construction calculates distances as needed, and hence there is no need to ever store the inter-point distance matrix. The recursive partitioning algorithm requires us to determine the vertices (and edges) of $T_L$ and $T_R$. We show how this can be done easily and efficiently using information generated by Prim's algorithm without any additional computational cost.