Exploiting Network Structures to Improve Semantic Representation for the Financial Domain
This work addresses the challenge of improving semantic representation specifically for the financial domain, but it is incremental as it builds on existing methods like BERT and knowledge graphs.
The paper tackled the problem of learning semantic similarities for financial domain entities by combining contextual embeddings from transformer models with network structure embeddings from knowledge graphs, achieving superior results with the knowledge graph-enhanced model and additional benefit from a proposed voting function.
This paper presents the participation of the MiniTrue team in the FinSim-3 shared task on learning semantic similarities for the financial domain in English language. Our approach combines contextual embeddings learned by transformer-based language models with network structures embeddings extracted on external knowledge sources, to create more meaningful representations of financial domain entities and terms. For this, two BERT based language models and a knowledge graph embedding model are used. Besides, we propose a voting function to joint three basic models for the final inference. Experimental results show that the model with the knowledge graph embeddings has achieved a superior result than these models with only contextual embeddings. Nevertheless, we also observe that our voting function brings an extra benefit to the final system.