Enriching Conversation Context in Retrieval-based Chatbots
This work addresses the efficiency and effectiveness of chatbots for users, but it is incremental as it builds on existing transformer-based methods.
The paper tackles the performance-speed trade-off in retrieval-based chatbots by developing a Bi-encoder architecture that uses the entire training set as a knowledge-base during inference, improving performance while maintaining high inference speed.
Work on retrieval-based chatbots, like most sequence pair matching tasks, can be divided into Cross-encoders that perform word matching over the pair, and Bi-encoders that encode the pair separately. The latter has better performance, however since candidate responses cannot be encoded offline, it is also much slower. Lately, multi-layer transformer architectures pre-trained as language models have been used to great effect on a variety of natural language processing and information retrieval tasks. Recent work has shown that these language models can be used in text-matching scenarios to create Bi-encoders that perform almost as well as Cross-encoders while having a much faster inference speed. In this paper, we expand upon this work by developing a sequence matching architecture that %takes into account contexts in the training dataset at inference time. utilizes the entire training set as a makeshift knowledge-base during inference. We perform detailed experiments demonstrating that this architecture can be used to further improve Bi-encoders performance while still maintaining a relatively high inference speed.