BERT: A Review of Applications in Natural Language Processing and Understanding
This is an incremental review paper that synthesizes existing research on BERT for students and researchers interested in natural language text analysis.
This paper reviews the BERT language model, describing its operation mechanisms, applications in text analytics tasks, comparisons with similar models, and proprietary variants, based on systematizing data from dozens of recent scientific articles.
In this review, we describe the application of one of the most popular deep learning-based language models - BERT. The paper describes the mechanism of operation of this model, the main areas of its application to the tasks of text analytics, comparisons with similar models in each task, as well as a description of some proprietary models. In preparing this review, the data of several dozen original scientific articles published over the past few years, which attracted the most attention in the scientific community, were systematized. This survey will be useful to all students and researchers who want to get acquainted with the latest advances in the field of natural language text analysis.