Unsupervised Joint $k$-node Graph Representations with Compositional Energy-Based Models
This work addresses a limitation in graph representation learning for tasks requiring larger node sets, though it appears incremental as it builds on existing GNN and energy-based model frameworks.
The paper tackles the problem of learning joint k-node graph representations for k>2, which existing GNN methods fail to do, and proposes MHM-GNN, an unsupervised approach combining energy-based models and GNNs with a tractable loss estimator, resulting in better unsupervised representations than existing methods.
Existing Graph Neural Network (GNN) methods that learn inductive unsupervised graph representations focus on learning node and edge representations by predicting observed edges in the graph. Although such approaches have shown advances in downstream node classification tasks, they are ineffective in jointly representing larger $k$-node sets, $k{>}2$. We propose MHM-GNN, an inductive unsupervised graph representation approach that combines joint $k$-node representations with energy-based models (hypergraph Markov networks) and GNNs. To address the intractability of the loss that arises from this combination, we endow our optimization with a loss upper bound using a finite-sample unbiased Markov Chain Monte Carlo estimator. Our experiments show that the unsupervised MHM-GNN representations of MHM-GNN produce better unsupervised representations than existing approaches from the literature.