Pentagon at MEDIQA 2019: Multi-task Learning for Filtering and Re-ranking Answers using Language Inference and Question Entailment
This work addresses the problem of improving answer ranking in medical QA for practitioners, but it is incremental as it builds on existing pre-trained models and multi-task learning approaches.
The paper tackled the challenge of filtering and re-ranking answers in medical question answering by introducing an end-to-end system trained in a multi-task setting, achieving a Spearman's Rho of 0.338 and Mean Reciprocal Rank of 0.9622 on the ACL-BioNLP MediQA shared-task.
Parallel deep learning architectures like fine-tuned BERT and MT-DNN, have quickly become the state of the art, bypassing previous deep and shallow learning methods by a large margin. More recently, pre-trained models from large related datasets have been able to perform well on many downstream tasks by just fine-tuning on domain-specific datasets . However, using powerful models on non-trivial tasks, such as ranking and large document classification, still remains a challenge due to input size limitations of parallel architecture and extremely small datasets (insufficient for fine-tuning). In this work, we introduce an end-to-end system, trained in a multi-task setting, to filter and re-rank answers in the medical domain. We use task-specific pre-trained models as deep feature extractors. Our model achieves the highest Spearman's Rho and Mean Reciprocal Rank of 0.338 and 0.9622 respectively, on the ACL-BioNLP workshop MediQA Question Answering shared-task.