IndoLEM and IndoBERT: A Benchmark Dataset and Pre-trained Language Model for Indonesian NLP
This addresses the lack of standardized resources for Indonesian NLP, benefiting researchers and practitioners working with this widely spoken language.
The authors tackled the under-representation of Indonesian in NLP by releasing IndoLEM, a benchmark dataset with seven tasks, and IndoBERT, a pre-trained language model, which achieves state-of-the-art performance on most tasks.
Although the Indonesian language is spoken by almost 200 million people and the 10th most spoken language in the world, it is under-represented in NLP research. Previous work on Indonesian has been hampered by a lack of annotated datasets, a sparsity of language resources, and a lack of resource standardization. In this work, we release the IndoLEM dataset comprising seven tasks for the Indonesian language, spanning morpho-syntax, semantics, and discourse. We additionally release IndoBERT, a new pre-trained language model for Indonesian, and evaluate it over IndoLEM, in addition to benchmarking it against existing resources. Our experiments show that IndoBERT achieves state-of-the-art performance over most of the tasks in IndoLEM.