Improving Heterogeneous Graph Learning with Weighted Mixed-Curvature Product Manifold
This work addresses the challenge of capturing complex geometric structures in heterogeneous graphs for applications in representation learning, though it is incremental as it builds on existing product manifold approaches by introducing weighting.
The paper tackled the problem of limited capacity in Euclidean embedding spaces for representing graphs with varying structures by proposing WEIGHTED-PM, a method for learning embeddings in weighted product manifolds, which achieved better graph representations with lower geometric distortion and improved performance on downstream tasks like word similarity learning, top-k recommendation, and knowledge graph embedding.
In graph representation learning, it is important that the complex geometric structure of the input graph, e.g. hidden relations among nodes, is well captured in embedding space. However, standard Euclidean embedding spaces have a limited capacity in representing graphs of varying structures. A promising candidate for the faithful embedding of data with varying structure is product manifolds of component spaces of different geometries (spherical, hyperbolic, or euclidean). In this paper, we take a closer look at the structure of product manifold embedding spaces and argue that each component space in a product contributes differently to expressing structures in the input graph, hence should be weighted accordingly. This is different from previous works which consider the roles of different components equally. We then propose WEIGHTED-PM, a data-driven method for learning embedding of heterogeneous graphs in weighted product manifolds. Our method utilizes the topological information of the input graph to automatically determine the weight of each component in product spaces. Extensive experiments on synthetic and real-world graph datasets demonstrate that WEIGHTED-PM is capable of learning better graph representations with lower geometric distortion from input data, and performs better on multiple downstream tasks, such as word similarity learning, top-$k$ recommendation, and knowledge graph embedding.