Entailment-Preserving First-order Logic Representations in Natural Language Entailment
This addresses the problem of accurate logical representation in natural language processing for tasks like automated reasoning, though it is incremental as it builds on existing NL-to-FOL translation methods.
The paper tackles the challenge of generating first-order logic (FOL) representations from natural language entailment data to preserve entailment labels, proposing the EPF task and iterative learning-to-rank method, which improves the Entailment-Preserving Rate by 1.8-2.7% and EPR@16 by 17.4-20.6% over baselines.
First-order logic (FOL) can represent the logical entailment semantics of natural language (NL) sentences, but determining natural language entailment using FOL remains a challenge. To address this, we propose the Entailment-Preserving FOL representations (EPF) task and introduce reference-free evaluation metrics for EPF, the Entailment-Preserving Rate (EPR) family. In EPF, one should generate FOL representations from multi-premise natural language entailment data (e.g. EntailmentBank) so that the automatic prover's result preserves the entailment labels. Experiments show that existing methods for NL-to-FOL translation struggle in EPF. To this extent, we propose a training method specialized for the task, iterative learning-to-rank, which directly optimizes the model's EPR score through a novel scoring function and a learning-to-rank objective. Our method achieves a 1.8-2.7% improvement in EPR and a 17.4-20.6% increase in EPR@16 compared to diverse baselines in three datasets. Further analyses reveal that iterative learning-to-rank effectively suppresses the arbitrariness of FOL representation by reducing the diversity of predicate signatures, and maintains strong performance across diverse inference types and out-of-domain data.