MOSLD-Bench: Multilingual Open-Set Learning and Discovery Benchmark for Text Categorization
This addresses the problem of handling unknown classes in text classification for multilingual applications, though it is incremental as it builds on existing zero-shot learning setups.
The authors tackled the challenge of open-set learning and discovery in text categorization by introducing MOSLD-Bench, the first multilingual benchmark with 960K samples across 12 languages, and proposed a novel framework that integrates multiple stages for continuous discovery and learning of new classes.
Open-set learning and discovery (OSLD) is a challenging machine learning task in which samples from new (unknown) classes can appear at test time. It can be seen as a generalization of zero-shot learning, where the new classes are not known a priori, hence involving the active discovery of new classes. While zero-shot learning has been extensively studied in text classification, especially with the emergence of pre-trained language models, open-set learning and discovery is a comparatively new setup for the text domain. To this end, we introduce the first multilingual open-set learning and discovery (MOSLD) benchmark for text categorization by topic, comprising 960K data samples across 12 languages. To construct the benchmark, we (i) rearrange existing datasets and (ii) collect new data samples from the news domain. Moreover, we propose a novel framework for the OSLD task, which integrates multiple stages to continuously discover and learn new classes. We evaluate several language models, including our own, to obtain results that can be used as reference for future work. We release our benchmark at https://github.com/Adriana19Valentina/MOSLD-Bench.