PGMEL: Policy Gradient-based Generative Adversarial Network for Multimodal Entity Linking
This work addresses the problem of linking entities in multimodal data for applications in knowledge graphs, representing an incremental improvement by focusing on negative sample selection.
The paper tackles multimodal entity linking by proposing a generative adversarial network that uses policy gradient to generate high-quality negative samples, resulting in improved performance over state-of-the-art methods on datasets like Wiki-MEL, Richpedia-MEL, and WikiDiverse.
The task of entity linking, which involves associating mentions with their respective entities in a knowledge graph, has received significant attention due to its numerous potential applications. Recently, various multimodal entity linking (MEL) techniques have been proposed, targeted to learn comprehensive embeddings by leveraging both text and vision modalities. The selection of high-quality negative samples can potentially play a crucial role in metric/representation learning. However, to the best of our knowledge, this possibility remains unexplored in existing literature within the framework of MEL. To fill this gap, we address the multimodal entity linking problem in a generative adversarial setting where the generator is responsible for generating high-quality negative samples, and the discriminator is assigned the responsibility for the metric learning tasks. Since the generator is involved in generating samples, which is a discrete process, we optimize it using policy gradient techniques and propose a policy gradient-based generative adversarial network for multimodal entity linking (PGMEL). Experimental results based on Wiki-MEL, Richpedia-MEL and WikiDiverse datasets demonstrate that PGMEL learns meaningful representation by selecting challenging negative samples and outperforms state-of-the-art methods.