Subword ELMo
This work addresses a bottleneck in word representation for NLP practitioners, offering incremental gains over existing methods.
The paper tackled the problem of ELMo's reliance on character-level information for word representation by introducing ESuLMo, which uses subwords from unsupervised segmentation, resulting in enhanced performance on four NLP benchmark tasks with meaningful improvements over ELMo.
Embedding from Language Models (ELMo) has shown to be effective for improving many natural language processing (NLP) tasks, and ELMo takes character information to compose word representation to train language models.However, the character is an insufficient and unnatural linguistic unit for word representation.Thus we introduce Embedding from Subword-aware Language Models (ESuLMo) which learns word representation from subwords using unsupervised segmentation over words.We show that ESuLMo can enhance four benchmark NLP tasks more effectively than ELMo, including syntactic dependency parsing, semantic role labeling, implicit discourse relation recognition and textual entailment, which brings a meaningful improvement over ELMo.