MuLVE, A Multi-Language Vocabulary Evaluation Data Set
This provides a dataset for improving vocabulary learning feedback, but it is incremental as it applies existing methods to new data.
The authors tackled the lack of real-life user data in vocabulary evaluation systems by introducing MuLVE, a multi-language dataset with labeled user answers, and achieved >95.5% accuracy and F2-score by fine-tuning BERT models on it.
Vocabulary learning is vital to foreign language learning. Correct and adequate feedback is essential to successful and satisfying vocabulary training. However, many vocabulary and language evaluation systems perform on simple rules and do not account for real-life user learning data. This work introduces Multi-Language Vocabulary Evaluation Data Set (MuLVE), a data set consisting of vocabulary cards and real-life user answers, labeled indicating whether the user answer is correct or incorrect. The data source is user learning data from the Phase6 vocabulary trainer. The data set contains vocabulary questions in German and English, Spanish, and French as target language and is available in four different variations regarding pre-processing and deduplication. We experiment to fine-tune pre-trained BERT language models on the downstream task of vocabulary evaluation with the proposed MuLVE data set. The results provide outstanding results of > 95.5 accuracy and F2-score. The data set is available on the European Language Grid.