Unified BERT for Few-shot Natural Language Understanding
This work addresses the challenge of unifying NLU tasks for applications like insurance processing, though it appears incremental as it builds on existing BERT frameworks.
The paper tackles the problem of diverse output schemas in natural language understanding by proposing UBERT, a unified BERT-based model that uses a biaffine network to convert various tasks into a span-decoding approach, achieving first place in a 2022 AI competition for Chinese insurance few-shot multi-task challenges.
Even as pre-trained language models share a semantic encoder, natural language understanding suffers from a diversity of output schemas. In this paper, we propose UBERT, a unified bidirectional language understanding model based on BERT framework, which can universally model the training objects of different NLU tasks through a biaffine network. Specifically, UBERT encodes prior knowledge from various aspects, uniformly constructing learning representations across multiple NLU tasks, which is conducive to enhancing the ability to capture common semantic understanding. By using the biaffine to model scores pair of the start and end position of the original text, various classification and extraction structures can be converted into a universal, span-decoding approach. Experiments show that UBERT wins the first price in the 2022 AIWIN - World Artificial Intelligence Innovation Competition, Chinese insurance few-shot multi-task track, and realizes the unification of extensive information extraction and linguistic reasoning tasks.