Generating abbreviations using Google Books library
This work addresses text segmentation challenges in applied text processing systems, though it is incremental as it builds on existing corpus data.
The authors tackled the problem of creating a dictionary of abbreviations for Russian using the Google Books Ngram Corpus, resulting in a method that can be applied to any language and includes a model for evaluating extraction accuracy and fullness with statistical data provided.
The article describes the original method of creating a dictionary of abbreviations based on the Google Books Ngram Corpus. The dictionary of abbreviations is designed for Russian, yet as its methodology is universal it can be applied to any language. The dictionary can be used to define the function of the period during text segmentation in various applied systems of text processing. The article describes difficulties encountered in the process of its construction as well as the ways to overcome them. A model of evaluating a probability of first and second type errors (extraction accuracy and fullness) is constructed. Certain statistical data for the use of abbreviations are provided.