CloneBot: Personalized Dialogue-Response Predictions
This enables more human-like speech bots for live conversations, but it is incremental as it builds on existing transformer and clustering techniques.
The paper tackled the problem of predicting personalized dialogue responses by using a transformer-based model with dense-vector encoding clustering to retrieve relevant historical context, achieving state-of-the-art results on the Switchboard corpus.
Our project task was to create a model that, given a speaker ID, chat history, and an utterance query, can predict the response utterance in a conversation. The model is personalized for each speaker. This task can be a useful tool for building speech bots that talk in a human-like manner in a live conversation. Further, we succeeded at using dense-vector encoding clustering to be able to retrieve relevant historical dialogue context, a useful strategy for overcoming the input limitations of neural-based models when predictions require longer-term references from the dialogue history. In this paper, we have implemented a state-of-the-art model using pre-training and fine-tuning techniques built on transformer architecture and multi-headed attention blocks for the Switchboard corpus. We also show how efficient vector clustering algorithms can be used for real-time utterance predictions that require no training and therefore work on offline and encrypted message histories.