AspectMMKG: A Multi-modal Knowledge Graph with Aspect-aware Entities
This addresses the limitation in MMKGs for comprehensive entity understanding, though it is incremental as it builds on existing MMKG frameworks by adding aspect-aware data.
The paper tackles the problem of multi-modal knowledge graphs (MMKGs) neglecting the multi-aspect nature of entities by constructing AspectMMKG, the first MMKG with aspect-related images, containing 2,380 entities, 18,139 entity aspects, and 645,383 images, and shows it enables previous entity aspect linking models to achieve state-of-the-art performance.
Multi-modal knowledge graphs (MMKGs) combine different modal data (e.g., text and image) for a comprehensive understanding of entities. Despite the recent progress of large-scale MMKGs, existing MMKGs neglect the multi-aspect nature of entities, limiting the ability to comprehend entities from various perspectives. In this paper, we construct AspectMMKG, the first MMKG with aspect-related images by matching images to different entity aspects. Specifically, we collect aspect-related images from a knowledge base, and further extract aspect-related sentences from the knowledge base as queries to retrieve a large number of aspect-related images via an online image search engine. Finally, AspectMMKG contains 2,380 entities, 18,139 entity aspects, and 645,383 aspect-related images. We demonstrate the usability of AspectMMKG in entity aspect linking (EAL) downstream task and show that previous EAL models achieve a new state-of-the-art performance with the help of AspectMMKG. To facilitate the research on aspect-related MMKG, we further propose an aspect-related image retrieval (AIR) model, that aims to correct and expand aspect-related images in AspectMMKG. We train an AIR model to learn the relationship between entity image and entity aspect-related images by incorporating entity image, aspect, and aspect image information. Experimental results indicate that the AIR model could retrieve suitable images for a given entity w.r.t different aspects.