Recursive Neural Networks Can Learn Logical Semantics
This addresses the problem of whether neural networks can handle logical deduction in natural language for AI and NLP applications, but it is incremental as it builds on existing TreeRNN models.
The study investigated whether tree-structured recursive neural networks (TreeRNNs) can learn logical semantics by evaluating their ability to identify logical relationships like entailment and contradiction. The models performed competitively on SICK data and generalized well on simulated data, indicating they can learn suitable representations for logical inference in natural language.
Tree-structured recursive neural networks (TreeRNNs) for sentence meaning have been successful for many applications, but it remains an open question whether the fixed-length representations that they learn can support tasks as demanding as logical deduction. We pursue this question by evaluating whether two such models---plain TreeRNNs and tree-structured neural tensor networks (TreeRNTNs)---can correctly learn to identify logical relationships such as entailment and contradiction using these representations. In our first set of experiments, we generate artificial data from a logical grammar and use it to evaluate the models' ability to learn to handle basic relational reasoning, recursive structures, and quantification. We then evaluate the models on the more natural SICK challenge data. Both models perform competitively on the SICK data and generalize well in all three experiments on simulated data, suggesting that they can learn suitable representations for logical inference in natural language.