Chain of Condition: Construct, Verify and Solve Conditions for Conditional Question Answering
This addresses the problem of precisely handling conditional questions in QA systems, offering a novel method for a specific domain with incremental improvements over existing prompting techniques.
The paper tackles conditional question answering by proposing Chain of Condition, a prompting approach that identifies conditions, verifies them, and solves logical expressions to find answers and missing conditions, achieving new state-of-the-art results on benchmark datasets and enabling GPT models to outperform supervised models with few examples.
Conditional question answering (CQA) is an important task that aims to find probable answers and identify missing conditions. Existing approaches struggle with CQA due to two challenges: (1) precisely identifying necessary conditions and the logical relationship, and (2) verifying conditions to detect any that are missing. In this paper, we propose a novel prompting approach, Chain of condition, by first identifying all conditions and constructing their logical relationships explicitly according to the document, then verifying whether these conditions are satisfied, finally solving the logical expression to indicate any missing conditions and generating the answer accordingly. Experiments on two CQA benchmark datasets show our chain of condition outperforms existing prompting baselines, establishing a new state of the art. Furthermore, with only a few examples, our method can facilitate GPT-3.5-Turbo or GPT-4 to outperform all existing supervised models.