Morph-fitting: Fine-Tuning Word Vector Spaces with Simple Language-Specific Rules
This addresses the issue of language understanding systems misinterpreting words in morphologically rich languages, though it is incremental as it builds on existing vector space models.
The paper tackled the problem of inaccurate word representations in morphologically rich languages by proposing a morph-fitting procedure that injects morphological constraints, resulting in improved low-frequency word estimates and semantic quality across four languages, with large gains in dialogue state tracking.
Morphologically rich languages accentuate two properties of distributional vector space models: 1) the difficulty of inducing accurate representations for low-frequency word forms; and 2) insensitivity to distinct lexical relations that have similar distributional signatures. These effects are detrimental for language understanding systems, which may infer that 'inexpensive' is a rephrasing for 'expensive' or may not associate 'acquire' with 'acquires'. In this work, we propose a novel morph-fitting procedure which moves past the use of curated semantic lexicons for improving distributional vector spaces. Instead, our method injects morphological constraints generated using simple language-specific rules, pulling inflectional forms of the same word close together and pushing derivational antonyms far apart. In intrinsic evaluation over four languages, we show that our approach: 1) improves low-frequency word estimates; and 2) boosts the semantic quality of the entire word vector collection. Finally, we show that morph-fitted vectors yield large gains in the downstream task of dialogue state tracking, highlighting the importance of morphology for tackling long-tail phenomena in language understanding tasks.