On the cohesion and separability of average-link for hierarchical agglomerative clustering
This work addresses the need for more interpretable and effective clustering methods in data analysis, though it is incremental as it builds on existing theoretical frameworks.
The paper tackled the theoretical and empirical performance of average-link hierarchical agglomerative clustering in metric spaces, focusing on criteria for cohesion and separability, and found that average-link outperforms other methods like single-linkage and complete-linkage in these aspects.
Average-link is widely recognized as one of the most popular and effective methods for building hierarchical agglomerative clustering. The available theoretical analyses show that this method has a much better approximation than other popular heuristics, as single-linkage and complete-linkage, regarding variants of Dasgupta's cost function [STOC 2016]. However, these analyses do not separate average-link from a random hierarchy and they are not appealing for metric spaces since every hierarchical clustering has a 1/2 approximation with regard to the variant of Dasgupta's function that is employed for dissimilarity measures [Moseley and Yang 2020]. In this paper, we present a comprehensive study of the performance of average-link in metric spaces, regarding several natural criteria that capture separability and cohesion and are more interpretable than Dasgupta's cost function and its variants. We also present experimental results with real datasets that, together with our theoretical analyses, suggest that average-link is a better choice than other related methods when both cohesion and separability are important goals.