Multilingual Text Representation
It addresses the problem of ensuring fair and unified text representation across diverse languages for NLP researchers and practitioners, but it is incremental as it reviews existing progress rather than introducing new methods.
This survey examines the progression of multilingual text representation in NLP, highlighting the expansion of models to over 100 languages and their capabilities in tasks like natural language understanding and question-answering, while noting ongoing challenges in achieving equitable representation across languages and speakers.
Modern NLP breakthrough includes large multilingual models capable of performing tasks across more than 100 languages. State-of-the-art language models came a long way, starting from the simple one-hot representation of words capable of performing tasks like natural language understanding, common-sense reasoning, or question-answering, thus capturing both the syntax and semantics of texts. At the same time, language models are expanding beyond our known language boundary, even competitively performing over very low-resource dialects of endangered languages. However, there are still problems to solve to ensure an equitable representation of texts through a unified modeling space across language and speakers. In this survey, we shed light on this iterative progression of multilingual text representation and discuss the driving factors that ultimately led to the current state-of-the-art. Subsequently, we discuss how the full potential of language democratization could be obtained, reaching beyond the known limits and what is the scope of improvement in that space.