High Quality Embeddings for Horn Logic Reasoning
This work addresses the need for effective embeddings in neural-symbolic reasoning, offering incremental improvements for Horn logic reasoning tasks.
The paper introduces and evaluates methods for creating embeddings of logical statements to improve the efficiency of neural-guided Horn logic reasoning. The proposed embedding training approach using triplet loss with novel anchor and example generation strategies achieves better downstream reasoning performance.
Neural networks can be trained to rank the choices made by logical reasoners, resulting in more efficient searches for answers. A key step in this process is creating useful embeddings, i.e., numeric representations of logical statements. This paper introduces and evaluates several approaches to creating embeddings that result in better downstream results. We train embeddings using triplet loss, which requires examples consisting of an anchor, a positive example, and a negative example. We introduce three ideas: generating anchors that are more likely to have repeated terms, generating positive and negative examples in a way that ensures a good balance between easy, medium, and hard examples, and periodically emphasizing the hardest examples during training. We conduct several experiments to evaluate this approach, including a comparison of different embeddings across different knowledge bases, in an attempt to identify what characteristics make an embedding well-suited to a particular reasoning task.