Epitomic Variational Graph Autoencoder
This work addresses a known bottleneck in graph-structured data modeling for researchers and practitioners using variational autoencoders, representing an incremental improvement.
The paper tackles the over-pruning problem in variational graph autoencoders (VGAE), which limits learning capacity by causing latent variables to become inactive, and proposes Epitomic VGAE (EVGAE) to mitigate this issue, resulting in improved generative ability and outperforming VGAE on link prediction tasks in citation networks.
Variational autoencoder (VAE) is a widely used generative model for learning latent representations. Burda et al. in their seminal paper showed that learning capacity of VAE is limited by over-pruning. It is a phenomenon where a significant number of latent variables fail to capture any information about the input data and the corresponding hidden units become inactive. This adversely affects learning diverse and interpretable latent representations. As variational graph autoencoder (VGAE) extends VAE for graph-structured data, it inherits the over-pruning problem. In this paper, we adopt a model based approach and propose epitomic VGAE (EVGAE),a generative variational framework for graph datasets which successfully mitigates the over-pruning problem and also boosts the generative ability of VGAE. We consider EVGAE to consist of multiple sparse VGAE models, called epitomes, that are groups of latent variables sharing the latent space. This approach aids in increasing active units as epitomes compete to learn better representation of the graph data. We verify our claims via experiments on three benchmark datasets. Our experiments show that EVGAE has a better generative ability than VGAE. Moreover, EVGAE outperforms VGAE on link prediction task in citation networks.