Augmenting Neural Networks with First-order Logic
This addresses the challenge of incorporating structured knowledge into neural models for tasks like machine comprehension, benefiting researchers in AI and NLP, though it is incremental as it builds on existing neural and logical methods.
The paper tackles the problem of integrating declarative world knowledge into neural networks while maintaining end-to-end trainability, resulting in a framework that compiles logical statements into computation graphs to guide training and prediction, with experiments showing strong improvements over baselines, particularly in low-data regimes.
Today, the dominant paradigm for training neural networks involves minimizing task loss on a large dataset. Using world knowledge to inform a model, and yet retain the ability to perform end-to-end training remains an open question. In this paper, we present a novel framework for introducing declarative knowledge to neural network architectures in order to guide training and prediction. Our framework systematically compiles logical statements into computation graphs that augment a neural network without extra learnable parameters or manual redesign. We evaluate our modeling strategy on three tasks: machine comprehension, natural language inference, and text chunking. Our experiments show that knowledge-augmented networks can strongly improve over baselines, especially in low-data regimes.