Generative Pre-trained Transformer for Vietnamese Community-based COVID-19 Question Answering
This work addresses a gap in applying advanced language models to Vietnamese COVID-19 information retrieval, but it is incremental as it adapts existing methods to a new language and dataset.
The paper tackled the lack of GPT applications in Vietnamese by implementing GPT-2 for COVID-19 question answering, showing it outperforms other state-of-the-art models in this domain.
Recent studies have provided empirical evidence of the wide-ranging potential of Generative Pre-trained Transformer (GPT), a pretrained language model, in the field of natural language processing. GPT has been effectively employed as a decoder within state-of-the-art (SOTA) question answering systems, yielding exceptional performance across various tasks. However, the current research landscape concerning GPT's application in Vietnamese remains limited. This paper aims to address this gap by presenting an implementation of GPT-2 for community-based question answering specifically focused on COVID-19 related queries in Vietnamese. We introduce a novel approach by conducting a comparative analysis of different Transformers vs SOTA models in the community-based COVID-19 question answering dataset. The experimental findings demonstrate that the GPT-2 models exhibit highly promising outcomes, outperforming other SOTA models as well as previous community-based COVID-19 question answering models developed for Vietnamese.