Harnessing Deep Neural Networks with Logic Rules
This work addresses the challenge of making neural networks more interpretable and effective by combining them with logic rules, which is incremental as it builds on existing methods.
The authors tackled the problem of integrating structured logic rules into deep neural networks to improve interpretability and performance, achieving state-of-the-art or comparable results in sentiment analysis and named entity recognition with substantial improvements.
Combining deep neural networks with structured logic rules is desirable to harness flexibility and reduce uninterpretability of the neural models. We propose a general framework capable of enhancing various types of neural networks (e.g., CNNs and RNNs) with declarative first-order logic rules. Specifically, we develop an iterative distillation method that transfers the structured information of logic rules into the weights of neural networks. We deploy the framework on a CNN for sentiment analysis, and an RNN for named entity recognition. With a few highly intuitive rules, we obtain substantial improvements and achieve state-of-the-art or comparable results to previous best-performing systems.