Reduced Implication-bias Logic Loss for Neuro-Symbolic Learning
This addresses a specific bottleneck in neuro-symbolic systems for researchers, but it is incremental as it modifies existing loss functions rather than introducing a new paradigm.
The paper tackles the problem of implication bias in neuro-symbolic learning, where biased loss functions from fuzzy logic operators degrade performance, and proposes a reduced implication-bias logic loss (RILL) that achieves significant improvements, especially with incomplete knowledge bases and insufficient labeled data.
Integrating logical reasoning and machine learning by approximating logical inference with differentiable operators is a widely used technique in Neuro-Symbolic systems. However, some differentiable operators could bring a significant bias during backpropagation and degrade the performance of Neuro-Symbolic learning. In this paper, we reveal that this bias, named \textit{Implication Bias} is common in loss functions derived from fuzzy logic operators. Furthermore, we propose a simple yet effective method to transform the biased loss functions into \textit{Reduced Implication-bias Logic Loss (RILL)} to address the above problem. Empirical study shows that RILL can achieve significant improvements compared with the biased logic loss functions, especially when the knowledge base is incomplete, and keeps more robust than the compared methods when labelled data is insufficient.