Unsupervised Post-processing of Word Vectors via Conceptor Negation
This work addresses the need for better semantic representation in natural language processing, offering an incremental improvement for tasks like lexical evaluation and dialogue systems.
The paper tackled the problem of enriching semantic information in word vectors by introducing an unsupervised post-processing technique using matrix conceptors to suppress high-variance latent features, resulting in consistent outperformance of state-of-the-art alternatives on intrinsic lexical evaluation tasks and improved results in dialogue state tracking.
Word vectors are at the core of many natural language processing tasks. Recently, there has been interest in post-processing word vectors to enrich their semantic information. In this paper, we introduce a novel word vector post-processing technique based on matrix conceptors (Jaeger2014), a family of regularized identity maps. More concretely, we propose to use conceptors to suppress those latent features of word vectors having high variances. The proposed method is purely unsupervised: it does not rely on any corpus or external linguistic database. We evaluate the post-processed word vectors on a battery of intrinsic lexical evaluation tasks, showing that the proposed method consistently outperforms existing state-of-the-art alternatives. We also show that post-processed word vectors can be used for the downstream natural language processing task of dialogue state tracking, yielding improved results in different dialogue domains.