emLam -- a Hungarian Language Modeling baseline
This provides a foundational resource for researchers and practitioners working on Hungarian NLP, but it is incremental as it adapts existing methods to a new language.
The paper tackled the lack of documented baselines for Hungarian language modeling by evaluating various approaches on three publicly available Hungarian corpora, reporting perplexity values comparable to models on similar-sized English corpora and introducing a new, freely downloadable Hungarian benchmark corpus.
This paper aims to make up for the lack of documented baselines for Hungarian language modeling. Various approaches are evaluated on three publicly available Hungarian corpora. Perplexity values comparable to models of similar-sized English corpora are reported. A new, freely downloadable Hungar- ian benchmark corpus is introduced.