Active entailment encoding for explanation tree construction using parsimonious generation of hard negatives
This addresses the problem of reducing human effort in building explanation trees for AI researchers and practitioners, though it is incremental as it builds on existing entailment tree methods.
The paper tackles the laborious process of manually constructing entailment trees for explanation generation in question answering by framing it as active premise selection and fine-tuning Transformer models with controlled negatives, resulting in up to 20% improvement in premise selection efficiency.
Entailment trees have been proposed to simulate the human reasoning process of explanation generation in the context of open--domain textual question answering. However, in practice, manually constructing these explanation trees proves a laborious process that requires active human involvement. Given the complexity of capturing the line of reasoning from question to the answer or from claim to premises, the issue arises of how to assist the user in efficiently constructing multi--level entailment trees given a large set of available facts. In this paper, we frame the construction of entailment trees as a sequence of active premise selection steps, i.e., for each intermediate node in an explanation tree, the expert needs to annotate positive and negative examples of premise facts from a large candidate list. We then iteratively fine--tune pre--trained Transformer models with the resulting positive and tightly controlled negative samples and aim to balance the encoding of semantic relationships and explanatory entailment relationships. Experimental evaluation confirms the measurable efficiency gains of the proposed active fine--tuning method in facilitating entailment trees construction: up to 20\% improvement in explanatory premise selection when compared against several alternatives.