Comparison of Current Approaches to Lemmatization: A Case Study in Estonian
This work addresses lemmatization for Estonian, an incremental improvement in a domain-specific NLP task.
The study compared three lemmatization approaches for Estonian and found that a smaller generative model outperformed a pattern-based classification model, with small error overlaps suggesting ensemble methods could improve results.
This study evaluates three different lemmatization approaches to Estonian -- Generative character-level models, Pattern-based word-level classification models, and rule-based morphological analysis. According to our experiments, a significantly smaller Generative model consistently outperforms the Pattern-based classification model based on EstBERT. Additionally, we observe a relatively small overlap in errors made by all three models, indicating that an ensemble of different approaches could lead to improvements.