MUSE: Modularizing Unsupervised Sense Embeddings
This addresses the problem of word sense ambiguity for natural language processing applications, offering a modular approach that improves over prior integrated models.
The paper tackles word sense ambiguity by proposing MUSE, a modular unsupervised system that learns sense representations and selects senses given contexts, achieving state-of-the-art performance on synonym selection and contextual word similarities with linear-time sense selection.
This paper proposes to address the word sense ambiguity issue in an unsupervised manner, where word sense representations are learned along a word sense selection mechanism given contexts. Prior work focused on designing a single model to deliver both mechanisms, and thus suffered from either coarse-grained representation learning or inefficient sense selection. The proposed modular approach, MUSE, implements flexible modules to optimize distinct mechanisms, achieving the first purely sense-level representation learning system with linear-time sense selection. We leverage reinforcement learning to enable joint training on the proposed modules, and introduce various exploration techniques on sense selection for better robustness. The experiments on benchmark data show that the proposed approach achieves the state-of-the-art performance on synonym selection as well as on contextual word similarities in terms of MaxSimC.