Pretrained Transformers for Simple Question Answering over Knowledge Graphs
This work addresses question answering for knowledge graph users, but it is incremental as it applies an existing method (BERT) to a new dataset.
The paper tackled the problem of simple question answering over knowledge graphs by evaluating BERT and BiLSTM models, finding that BERT outperforms previous approaches in data-sparse scenarios.
Answering simple questions over knowledge graphs is a well-studied problem in question answering. Previous approaches for this task built on recurrent and convolutional neural network based architectures that use pretrained word embeddings. It was recently shown that finetuning pretrained transformer networks (e.g. BERT) can outperform previous approaches on various natural language processing tasks. In this work, we investigate how well BERT performs on SimpleQuestions and provide an evaluation of both BERT and BiLSTM-based models in datasparse scenarios.