Semantic Answer Type Prediction using BERT: IAI at the ISWC SMART Task 2020
This work addresses answer type prediction for question answering systems, showing incremental improvements in fine-grained classification using BERT.
The paper investigated the performance of BERT on answer type prediction, finding that coarse-grained types achieve over 95% accuracy with standard methods and BERT offers only marginal gains, while BERT clearly outperforms previous approaches for fine-grained type detection.
This paper summarizes our participation in the SMART Task of the ISWC 2020 Challenge. A particular question we are interested in answering is how well neural methods, and specifically transformer models, such as BERT, perform on the answer type prediction task compared to traditional approaches. Our main finding is that coarse-grained answer types can be identified effectively with standard text classification methods, with over 95% accuracy, and BERT can bring only marginal improvements. For fine-grained type detection, on the other hand, BERT clearly outperforms previous retrieval-based approaches.