Robust Bayesian Model Selection for Variable Clustering with the Gaussian Graphical Model
This addresses a robustness issue in variable clustering for explanatory analysis, offering an incremental improvement over existing methods like BIC.
The paper tackles the problem of variable clustering with Gaussian graphical models being sensitive to noise in partial correlations, proposing a Bayesian model that accounts for negligible correlations and uses marginal likelihood for evaluation. Experiments show the method matches BIC accuracy in no-noise settings and is considerably more accurate with noise, providing sensible results on real data.
Variable clustering is important for explanatory analysis. However, only few dedicated methods for variable clustering with the Gaussian graphical model have been proposed. Even more severe, small insignificant partial correlations due to noise can dramatically change the clustering result when evaluating for example with the Bayesian Information Criteria (BIC). In this work, we try to address this issue by proposing a Bayesian model that accounts for negligible small, but not necessarily zero, partial correlations. Based on our model, we propose to evaluate a variable clustering result using the marginal likelihood. To address the intractable calculation of the marginal likelihood, we propose two solutions: one based on a variational approximation, and another based on MCMC. Experiments on simulated data shows that the proposed method is similarly accurate as BIC in the no noise setting, but considerably more accurate when there are noisy partial correlations. Furthermore, on real data the proposed method provides clustering results that are intuitively sensible, which is not always the case when using BIC or its extensions.