Atomic Inference for NLI with Generated Facts as Atoms
This addresses interpretability issues in NLI models, though it appears incremental as it builds on existing atomic inference methods.
The paper tackles the problem of unfaithful explanations in neural models for Natural Language Inference by proposing atomic inference with LLM-generated facts as atoms, achieving improved performance through multi-stage fact generation and training incorporation.
With recent advances, neural models can achieve human-level performance on various natural language tasks. However, there are no guarantees that any explanations from these models are faithful, i.e. that they reflect the inner workings of the model. Atomic inference overcomes this issue, providing interpretable and faithful model decisions. This approach involves making predictions for different components (or atoms) of an instance, before using interpretable and deterministic rules to derive the overall prediction based on the individual atom-level predictions. We investigate the effectiveness of using LLM-generated facts as atoms, decomposing Natural Language Inference premises into lists of facts. While directly using generated facts in atomic inference systems can result in worse performance, with 1) a multi-stage fact generation process, and 2) a training regime that incorporates the facts, our fact-based method outperforms other approaches.