TensorLog: A Differentiable Deductive Database
This work addresses the challenge of incorporating symbolic knowledge into deep learning for AI researchers, though it is incremental as it builds on existing probabilistic logic methods.
The authors tackled the problem of integrating large knowledge bases into gradient-based learning systems by introducing TensorLog, a differentiable probabilistic deductive database that converts logical clauses into factor graphs and unrolls belief propagation into differentiable functions, achieving efficient compilation and inference with linear complexity.
Large knowledge bases (KBs) are useful in many tasks, but it is unclear how to integrate this sort of knowledge into "deep" gradient-based learning systems. To address this problem, we describe a probabilistic deductive database, called TensorLog, in which reasoning uses a differentiable process. In TensorLog, each clause in a logical theory is first converted into certain type of factor graph. Then, for each type of query to the factor graph, the message-passing steps required to perform belief propagation (BP) are "unrolled" into a function, which is differentiable. We show that these functions can be composed recursively to perform inference in non-trivial logical theories containing multiple interrelated clauses and predicates. Both compilation and inference in TensorLog are efficient: compilation is linear in theory size and proof depth, and inference is linear in database size and the number of message-passing steps used in BP. We also present experimental results with TensorLog and discuss its relationship to other first-order probabilistic logics.