Cats climb entails mammals move: preserving hyponymy in compositional distributional semantics
This work addresses a specific challenge in natural language processing for representing meaning with structured vector spaces, but it is incremental as it builds on existing psd matrix approaches and shows limited experimental gains.
The paper tackles the problem of preserving hyponymy relationships in compositional distributional semantics by introducing a new composition rule called Compr, which uses completely positive maps to lift positive semidefinite matrices, resulting in improved performance on a small sentence entailment dataset compared to previous methods like Fuzz and Phaser.
To give vector-based representations of meaning more structure, one approach is to use positive semidefinite (psd) matrices. These allow us to model similarity of words as well as the hyponymy or is-a relationship. Psd matrices can be learnt relatively easily in a given vector space $M\otimes M^*$, but to compose words to form phrases and sentences, we need representations in larger spaces. In this paper, we introduce a generic way of composing the psd matrices corresponding to words. We propose that psd matrices for verbs, adjectives, and other functional words be lifted to completely positive (CP) maps that match their grammatical type. This lifting is carried out by our composition rule called Compression, Compr. In contrast to previous composition rules like Fuzz and Phaser (a.k.a. KMult and BMult), Compr preserves hyponymy. Mathematically, Compr is itself a CP map, and is therefore linear and generally non-commutative. We give a number of proposals for the structure of Compr, based on spiders, cups and caps, and generate a range of composition rules. We test these rules on a small sentence entailment dataset, and see some improvements over the performance of Fuzz and Phaser.