DC-BERT: Decoupling Question and Document for Efficient Contextual Encoding
This addresses the throughput bottleneck for high-volume question answering systems, though it is incremental as it builds on existing BERT-based methods.
The paper tackled the efficiency problem in open-domain question answering by proposing DC-BERT, a decoupled contextual encoding framework that achieves a 10x speedup in document retrieval while retaining about 98% of QA performance on SQuAD Open and Natural Questions Open datasets.
Recent studies on open-domain question answering have achieved prominent performance improvement using pre-trained language models such as BERT. State-of-the-art approaches typically follow the "retrieve and read" pipeline and employ BERT-based reranker to filter retrieved documents before feeding them into the reader module. The BERT retriever takes as input the concatenation of question and each retrieved document. Despite the success of these approaches in terms of QA accuracy, due to the concatenation, they can barely handle high-throughput of incoming questions each with a large collection of retrieved documents. To address the efficiency problem, we propose DC-BERT, a decoupled contextual encoding framework that has dual BERT models: an online BERT which encodes the question only once, and an offline BERT which pre-encodes all the documents and caches their encodings. On SQuAD Open and Natural Questions Open datasets, DC-BERT achieves 10x speedup on document retrieval, while retaining most (about 98%) of the QA performance compared to state-of-the-art approaches for open-domain question answering.