Scalable Weibull Graph Attention Autoencoder for Modeling Document Networks
This work addresses the challenge of generating hierarchical latent representations for interconnected documents, which is incremental as it builds on and combines existing VGAE and RTM methods.
The paper tackled the problem of modeling document relational networks (DRNs) with existing variational graph autoencoders (VGAEs) by incorporating relational topic models (RTMs) to improve flexibility for sparse and skewed latent representations, resulting in the development of Weibull graph auto-encoders (WGAEs) that achieve promising performance on graph analytic tasks.
Although existing variational graph autoencoders (VGAEs) have been widely used for modeling and generating graph-structured data, most of them are still not flexible enough to approximate the sparse and skewed latent node representations, especially those of document relational networks (DRNs) with discrete observations. To analyze a collection of interconnected documents, a typical branch of Bayesian models, specifically relational topic models (RTMs), has proven their efficacy in describing both link structures and document contents of DRNs, which motives us to incorporate RTMs with existing VGAEs to alleviate their potential issues when modeling the generation of DRNs. In this paper, moving beyond the sophisticated approximate assumptions of traditional RTMs, we develop a graph Poisson factor analysis (GPFA), which provides analytic conditional posteriors to improve the inference accuracy, and extend GPFA to a multi-stochastic-layer version named graph Poisson gamma belief network (GPGBN) to capture the hierarchical document relationships at multiple semantic levels. Then, taking GPGBN as the decoder, we combine it with various Weibull-based graph inference networks, resulting in two variants of Weibull graph auto-encoder (WGAE), equipped with model inference algorithms. Experimental results demonstrate that our models can extract high-quality hierarchical latent document representations and achieve promising performance on various graph analytic tasks.