Compositionality for Recursive Neural Networks
This work addresses a computational bottleneck for researchers in vector space semantics and neural networks, offering a way to make categorical models more practical.
The paper tackles the computational infeasibility of categorical compositional vector space models, which require high-dimensional matrices and tensors, by showing that a linear simplification of recursive neural tensor networks can be mapped onto this approach to compute these structures.
Modelling compositionality has been a longstanding area of research in the field of vector space semantics. The categorical approach to compositionality maps grammar onto vector spaces in a principled way, but comes under fire for requiring the formation of very high-dimensional matrices and tensors, and therefore being computationally infeasible. In this paper I show how a linear simplification of recursive neural tensor network models can be mapped directly onto the categorical approach, giving a way of computing the required matrices and tensors. This mapping suggests a number of lines of research for both categorical compositional vector space models of meaning and for recursive neural network models of compositionality.