Injecting Domain Knowledge in Neural Networks: a Controlled Experiment on a Constrained Problem
This work addresses the challenge of applying neural networks in domains with limited data but abundant expert knowledge, such as constraint problems, though it is incremental as it builds on existing ideas of knowledge injection.
The paper tackles the problem of data scarcity in deep learning by investigating how injecting domain knowledge into neural networks can ease learning, using a constrained problem as a case study. The results show that embedding domain knowledge at training time has a considerable effect, with a small amount of empirical knowledge yielding practically useful outcomes.
Given enough data, Deep Neural Networks (DNNs) are capable of learning complex input-output relations with high accuracy. In several domains, however, data is scarce or expensive to retrieve, while a substantial amount of expert knowledge is available. It seems reasonable that if we can inject this additional information in the DNN, we could ease the learning process. One such case is that of Constraint Problems, for which declarative approaches exists and pure ML solutions have obtained mixed success. Using a classical constrained problem as a case study, we perform controlled experiments to probe the impact of progressively adding domain and empirical knowledge in the DNN. Our results are very encouraging, showing that (at least in our setup) embedding domain knowledge at training time can have a considerable effect and that a small amount of empirical knowledge is sufficient to obtain practically useful results.