Discriminating Similar Languages: Evaluations and Explorations
This work addresses the problem of language discrimination for NLP applications, but it is incremental as it builds on existing shared tasks without introducing new methods.
The paper analyzed machine learning classifiers for discriminating similar languages and varieties, using data from two DSL shared tasks to evaluate progress, estimate performance bounds via ensembles, and identify challenging languages and sentences.
We present an analysis of the performance of machine learning classifiers on discriminating between similar languages and language varieties. We carried out a number of experiments using the results of the two editions of the Discriminating between Similar Languages (DSL) shared task. We investigate the progress made between the two tasks, estimate an upper bound on possible performance using ensemble and oracle combination, and provide learning curves to help us understand which languages are more challenging. A number of difficult sentences are identified and investigated further with human annotation.