Comparative Analysis of Multilingual Text Classification & Identification through Deep Learning and Embedding Visualization
It provides incremental insights for practitioners developing language detection and classification systems by evaluating existing techniques.
This study compared multilingual text classification methods using deep learning and embedding visualization on a dataset of 17 languages, finding that FastText with a multi-layer perceptron achieved high accuracy, precision, recall, and F1 score, outperforming Sentence Transformer models.
This research conducts a comparative study on multilingual text classification methods, utilizing deep learning and embedding visualization. The study employs LangDetect, LangId, FastText, and Sentence Transformer on a dataset encompassing 17 languages. It explores dimensionality's impact on clustering, revealing FastText's clearer clustering in 2D visualization due to its extensive multilingual corpus training. Notably, the FastText multi-layer perceptron model achieved remarkable accuracy, precision, recall, and F1 score, outperforming the Sentence Transformer model. The study underscores the effectiveness of these techniques in multilingual text classification, emphasizing the importance of large multilingual corpora for training embeddings. It lays the groundwork for future research and assists practitioners in developing language detection and classification systems. Additionally, it includes the comparison of multi-layer perceptron, LSTM, and Convolution models for classification.