QBERT: Generalist Model for Processing Questions
This work addresses the need for versatile models in natural language processing, but it is incremental as it builds on existing multi-task learning approaches without introducing a new paradigm.
The paper tackles the problem of developing generalist text representations for multiple question-related tasks, and demonstrates that their multi-task model QBERT achieves similar performance to single-task models.
Using a single model across various tasks is beneficial for training and applying deep neural sequence models. We address the problem of developing generalist representations of text that can be used to perform a range of different tasks rather than being specialised to a single application. We focus on processing short questions and developing an embedding for these questions that is useful on a diverse set of problems, such as question topic classification, equivalent question recognition, and question answering. This paper introduces QBERT, a generalist model for processing questions. With QBERT, we demonstrate how we can train a multi-task network that performs all question-related tasks and has achieved similar performance compared to its corresponding single-task models.