Hongzhu Li

Hongzhu Li, Xiangrui Gao, Linhui Feng et al.

Conventional representation learning algorithms for knowledge graphs (KG) map each entity to a unique embedding vector, ignoring the rich information contained in the neighborhood. We propose a method named StarGraph, which gives a novel way to utilize the neighborhood information for large-scale knowledge graphs to obtain entity representations. An incomplete two-hop neighborhood subgraph for each target node is at first generated, then processed by a modified self-attention network to obtain the entity representation, which is used to replace the entity embedding in conventional methods. We achieved SOTA performance on ogbl-wikikg2 and got competitive results on fb15k-237. The experimental results proves that StarGraph is efficient in parameters, and the improvement made on ogbl-wikikg2 demonstrates its great effectiveness of representation learning on large-scale knowledge graphs. The code is now available at \url{https://github.com/hzli-ucas/StarGraph}.

Long Yu, Zhicong Luo, Huanyong Liu et al.

Translation-based knowledge graph embedding has been one of the most important branches for knowledge representation learning since TransE came out. Although many translation-based approaches have achieved some progress in recent years, the performance was still unsatisfactory. This paper proposes a novel knowledge graph embedding method named TripleRE with two versions. The first version of TripleRE creatively divide the relationship vector into three parts. The second version takes advantage of the concept of residual and achieves better performance. In addition, attempts on using NodePiece to encode entities achieved promising results in reducing the parametric size, and solved the problems of scalability. Experiments show that our approach achieved state-of-the-art performance on the large-scale knowledge graph dataset, and competitive performance on other datasets.

Hongzhu Li, Weiqiang Wang

The connectionist temporal classification (CTC) enables end-to-end sequence learning by maximizing the probability of correctly recognizing sequences during training. The outputs of a CTC-trained model tend to form a series of spikes separated by strongly predicted blanks, know as the spiky problem. To figure out the reason for it, we reinterpret the CTC training process as an iterative fitting task that is based on frame-wise cross-entropy loss. It offers us an intuitive way to compare target probabilities with model outputs for each iteration, and explain how the model outputs gradually turns spiky. Inspired by it, we put forward two ways to modify the CTC training. The experiments demonstrate that our method can well solve the spiky problem and moreover, lead to faster convergence over various training settings. Beside this, the reinterpretation of CTC, as a brand new perspective, may be potentially useful in other situations. The code is publicly available at https://github.com/hzli-ucas/caffe/tree/ctc.