Image-embodied Knowledge Representation Learning
This work addresses the limitation of ignoring visual data in knowledge representation learning, which could benefit applications in AI domains like semantic search and recommendation systems, though it is incremental as it builds on existing methods by adding image integration.
The authors tackled the problem of knowledge representation learning by incorporating visual information from entity images, in addition to structured triples, and achieved improved performance on knowledge graph completion and triple classification tasks, outperforming all baselines.
Entity images could provide significant visual information for knowledge representation learning. Most conventional methods learn knowledge representations merely from structured triples, ignoring rich visual information extracted from entity images. In this paper, we propose a novel Image-embodied Knowledge Representation Learning model (IKRL), where knowledge representations are learned with both triple facts and images. More specifically, we first construct representations for all images of an entity with a neural image encoder. These image representations are then integrated into an aggregated image-based representation via an attention-based method. We evaluate our IKRL models on knowledge graph completion and triple classification. Experimental results demonstrate that our models outperform all baselines on both tasks, which indicates the significance of visual information for knowledge representations and the capability of our models in learning knowledge representations with images.