Transformers on Multilingual Clause-Level Morphology
This work addresses morphological processing for multiple languages, but it is incremental as it applies existing methods to a new shared task.
The authors tackled multilingual clause-level morphology tasks (inflection, reinflection, and analysis) by exploring transformers with data augmentation and prefix-tuning, achieving first place in all three tasks at the MRL 2022 shared task.
This paper describes our winning systems in MRL: The 1st Shared Task on Multilingual Clause-level Morphology (EMNLP 2022 Workshop) designed by KUIS AI NLP team. We present our work for all three parts of the shared task: inflection, reinflection, and analysis. We mainly explore transformers with two approaches: (i) training models from scratch in combination with data augmentation, and (ii) transfer learning with prefix-tuning at multilingual morphological tasks. Data augmentation significantly improves performance for most languages in the inflection and reinflection tasks. On the other hand, Prefix-tuning on a pre-trained mGPT model helps us to adapt analysis tasks in low-data and multilingual settings. While transformer architectures with data augmentation achieved the most promising results for inflection and reinflection tasks, prefix-tuning on mGPT received the highest results for the analysis task. Our systems received 1st place in all three tasks in MRL 2022.