Adaptive Joint Learning of Compositional and Non-Compositional Phrase Embeddings
This work addresses the challenge of accurately representing phrases in natural language processing, particularly for tasks like disambiguation, though it is incremental in nature.
The paper tackled the problem of learning phrase embeddings by jointly modeling compositional and non-compositional phrases using an adaptive weighting method, resulting in strong correlation with human ratings for verb-object compositionality and improved performance on a transitive verb disambiguation task.
We present a novel method for jointly learning compositional and non-compositional phrase embeddings by adaptively weighting both types of embeddings using a compositionality scoring function. The scoring function is used to quantify the level of compositionality of each phrase, and the parameters of the function are jointly optimized with the objective for learning phrase embeddings. In experiments, we apply the adaptive joint learning method to the task of learning embeddings of transitive verb phrases, and show that the compositionality scores have strong correlation with human ratings for verb-object compositionality, substantially outperforming the previous state of the art. Moreover, our embeddings improve upon the previous best model on a transitive verb disambiguation task. We also show that a simple ensemble technique further improves the results for both tasks.