Improving Continual Learning of Knowledge Graph Embeddings via Informed Initialization
This work addresses the challenge of efficiently updating knowledge graph embeddings for frequently changing KGs, offering an incremental improvement to existing continual learning methods.
The paper tackles the problem of initializing embeddings for new entities in continual learning for knowledge graphs, proposing an informed initialization strategy that uses the KG schema and previous embeddings to improve predictive performance and reduce training time. Experimental results show enhanced knowledge retention and faster acquisition, with benefits across various KGE models.
Many Knowledege Graphs (KGs) are frequently updated, forcing their Knowledge Graph Embeddings (KGEs) to adapt to these changes. To address this problem, continual learning techniques for KGEs incorporate embeddings for new entities while updating the old ones. One necessary step in these methods is the initialization of the embeddings, as an input to the KGE learning process, which can have an important impact in the accuracy of the final embeddings, as well as in the time required to train them. This is especially relevant for relatively small and frequent updates. We propose a novel informed embedding initialization strategy, which can be seamlessly integrated into existing continual learning methods for KGE, that enhances the acquisition of new knowledge while reducing catastrophic forgetting. Specifically, the KG schema and the previously learned embeddings are utilized to obtain initial representations for the new entities, based on the classes the entities belong to. Our extensive experimental analysis shows that the proposed initialization strategy improves the predictive performance of the resulting KGEs, while also enhancing knowledge retention. Furthermore, our approach accelerates knowledge acquisition, reducing the number of epochs, and therefore time, required to incrementally learn new embeddings. Finally, its benefits across various types of KGE learning models are demonstrated.