Ontology-Aware Token Embeddings for Prepositional Phrase Attachment
This work addresses lexical ambiguity in natural language processing for tasks like prepositional phrase attachment, but it is incremental as it builds on existing embedding and WordNet-based methods.
The paper tackled the problem of lexical ambiguity in type-level word embeddings by embedding semantic concepts from WordNet and using context-sensitive distributions for word tokens, resulting in a 5.4% absolute improvement in prepositional phrase attachment accuracy, which corresponds to a 34.4% relative error reduction.
Type-level word embeddings use the same set of parameters to represent all instances of a word regardless of its context, ignoring the inherent lexical ambiguity in language. Instead, we embed semantic concepts (or synsets) as defined in WordNet and represent a word token in a particular context by estimating a distribution over relevant semantic concepts. We use the new, context-sensitive embeddings in a model for predicting prepositional phrase(PP) attachments and jointly learn the concept embeddings and model parameters. We show that using context-sensitive embeddings improves the accuracy of the PP attachment model by 5.4% absolute points, which amounts to a 34.4% relative reduction in errors.