Implementation of an Index Optimize Technology for Highly Specialized Terms based on the Phonetic Algorithm Metaphone
This work addresses data deduplication issues in healthcare and other databases for non-English names, but it is incremental as it adapts an existing algorithm to specific linguistic contexts.
The paper tackles the problem of indexing and searching highly specialized names in databases, particularly for Ukrainian and Russian surnames, by adapting the Metaphone phonetic algorithm to reduce input errors and improve fuzzy matching, achieving unspecified reductions in duplication errors.
When compiling databases, for example to meet the needs of healthcare establishments, there is quite a common problem with the introduction and further processing of names and last names of doctors and patients that are highly specialized both in terms of pronunciation and writing. This is because names and last names of people cannot be unique, their notation is not subject to any rules of phonetics, while their length in different languages may not match. With the advent of the Internet, this situation has become generally critical and can lead to that multiple copies of e-mails are sent to one address. It is possible to solve the specified problem by using phonetic algorithms for comparing words Daitch-Mokotoff, Soundex, NYSIIS, Polyphone, and Metaphone, as well as the Levenshtein and Jaro algorithms, Q-gram-based algorithms, which make it possible to find distances between words. The most widespread among them are the Soundex and Metaphone algorithms, which are designed to index the words based on their sound, taking into consideration the rules of pronunciation. By applying the Metaphone algorithm, an attempt has been made to optimize the phonetic search processes for tasks of fuzzy coincidence, for example, at data deduplication in various databases and registries, in order to reduce the number of errors of incorrect input of last names. An analysis of the most common last names reveals that some of them are of the Ukrainian or Russian origin. At the same time, the rules following which the names are pronounced and written, for example in Ukrainian, differ radically from basic algorithms for English and differ quite significantly for the Russian language. That is why a phonetic algorithm should take into consideration first of all the peculiarities in the formation of Ukrainian last names, which is of special relevance now.