On the Model Shrinkage Effect of Gamma Process Edge Partition Models
This addresses a specific issue in Bayesian nonparametric models for overlapping structure extraction, offering incremental improvements for researchers in machine learning and statistics.
The paper tackled the problem of the edge partition model (EPM) overfitting due to ineffective model shrinkage with gamma process priors, and proposed two new models (CEPM and DEPM) that improved shrinkage, with IDEPM achieving state-of-the-art performance in generalization, link prediction, mixing efficiency, and convergence speed.
The edge partition model (EPM) is a fundamental Bayesian nonparametric model for extracting an overlapping structure from binary matrix. The EPM adopts a gamma process ($Γ$P) prior to automatically shrink the number of active atoms. However, we empirically found that the model shrinkage of the EPM does not typically work appropriately and leads to an overfitted solution. An analysis of the expectation of the EPM's intensity function suggested that the gamma priors for the EPM hyperparameters disturb the model shrinkage effect of the internal $Γ$P. In order to ensure that the model shrinkage effect of the EPM works in an appropriate manner, we proposed two novel generative constructions of the EPM: CEPM incorporating constrained gamma priors, and DEPM incorporating Dirichlet priors instead of the gamma priors. Furthermore, all DEPM's model parameters including the infinite atoms of the $Γ$P prior could be marginalized out, and thus it was possible to derive a truly infinite DEPM (IDEPM) that can be efficiently inferred using a collapsed Gibbs sampler. We experimentally confirmed that the model shrinkage of the proposed models works well and that the IDEPM indicated state-of-the-art performance in generalization ability, link prediction accuracy, mixing efficiency, and convergence speed.