Paraphrase-Supervised Models of Compositionality
This work addresses the challenge of semantic compositionality in natural language processing for applications like machine translation, but it is incremental as it builds on existing techniques with new supervision sources.
The paper tackles the problem of training compositional vector space models for language understanding by using automatically-extracted paraphrase examples as supervision, replacing manual annotations, and develops a context-aware model for scoring phrasal compositionality. The result shows that these approaches match previous techniques in intrinsic tasks and improve translation quality in a machine translation system.
Compositional vector space models of meaning promise new solutions to stubborn language understanding problems. This paper makes two contributions toward this end: (i) it uses automatically-extracted paraphrase examples as a source of supervision for training compositional models, replacing previous work which relied on manual annotations used for the same purpose, and (ii) develops a context-aware model for scoring phrasal compositionality. Experimental results indicate that these multiple sources of information can be used to learn partial semantic supervision that matches previous techniques in intrinsic evaluation tasks. Our approaches are also evaluated for their impact on a machine translation system where we show improvements in translation quality, demonstrating that compositionality in interpretation correlates with compositionality in translation.