Semantic Composition via Probabilistic Model Theory
This work addresses the challenge of modeling context-dependent meanings in formal semantics and composing distributed representations in distributional semantics, offering a bidirectional connection between these fields.
The paper tackles the problem of semantic composition in vector space models by interpreting a probabilistic graphical model from Functional Distributional Semantics as a probabilistic version of model theory, and shows that this approach improves performance on two datasets beyond word similarity.
Semantic composition remains an open problem for vector space models of semantics. In this paper, we explain how the probabilistic graphical model used in the framework of Functional Distributional Semantics can be interpreted as a probabilistic version of model theory. Building on this, we explain how various semantic phenomena can be recast in terms of conditional probabilities in the graphical model. This connection between formal semantics and machine learning is helpful in both directions: it gives us an explicit mechanism for modelling context-dependent meanings (a challenge for formal semantics), and also gives us well-motivated techniques for composing distributed representations (a challenge for distributional semantics). We present results on two datasets that go beyond word similarity, showing how these semantically-motivated techniques improve on the performance of vector models.