Bayesian Hierarchical Clustering with Exponential Family: Small-Variance Asymptotics and Reducibility
This work addresses scalability challenges in clustering for data scientists, though it is incremental as it builds on existing Bayesian hierarchical clustering methods.
The paper tackled the scalability and hyperparameter tuning issues in Bayesian hierarchical clustering by developing a relaxed non-probabilistic formulation using small-variance asymptotics, resulting in a method that matches the scalability of distance-based algorithms while maintaining flexibility, with numerical experiments showing high performance on synthetic and real-world datasets.
Bayesian hierarchical clustering (BHC) is an agglomerative clustering method, where a probabilistic model is defined and its marginal likelihoods are evaluated to decide which clusters to merge. While BHC provides a few advantages over traditional distance-based agglomerative clustering algorithms, successive evaluation of marginal likelihoods and careful hyperparameter tuning are cumbersome and limit the scalability. In this paper we relax BHC into a non-probabilistic formulation, exploring small-variance asymptotics in conjugate-exponential models. We develop a novel clustering algorithm, referred to as relaxed BHC (RBHC), from the asymptotic limit of the BHC model that exhibits the scalability of distance-based agglomerative clustering algorithms as well as the flexibility of Bayesian nonparametric models. We also investigate the reducibility of the dissimilarity measure emerged from the asymptotic limit of the BHC model, allowing us to use scalable algorithms such as the nearest neighbor chain algorithm. Numerical experiments on both synthetic and real-world datasets demonstrate the validity and high performance of our method.