Splitting Compounds by Semantic Analogy
This work addresses a specific issue in natural language processing for languages with productive compounding, offering an incremental improvement over existing methods.
The paper tackles the problem of compound word processing in NLP by using distributional semantics and semantic analogies to improve compound splitting, resulting in better machine translation quality compared to a frequency-based method.
Compounding is a highly productive word-formation process in some languages that is often problematic for natural language processing applications. In this paper, we investigate whether distributional semantics in the form of word embeddings can enable a deeper, i.e., more knowledge-rich, processing of compounds than the standard string-based methods. We present an unsupervised approach that exploits regularities in the semantic vector space (based on analogies such as "bookshop is to shop as bookshelf is to shelf") to produce compound analyses of high quality. A subsequent compound splitting algorithm based on these analyses is highly effective, particularly for ambiguous compounds. German to English machine translation experiments show that this semantic analogy-based compound splitter leads to better translations than a commonly used frequency-based method.