Chain-of-Questions Training with Latent Answers for Robust Multistep Question Answering
This addresses the problem of robust multistep question answering for AI systems, offering a novel method for handling latent variables in training.
The paper tackles robust multistep question answering by training a language model to generate and answer sub-questions, using a framework called Chain-of-Questions that treats sub-answers as latent variables optimized with a novel dynamic mixture of Hard-EM and MAPO. It achieves a 9.0 F1 improvement over strong neuro-symbolic methods on DROP contrast set and a 24.3 F1 improvement over GPT-3.5 on HOTPOTQA adversarial set.
We train a language model (LM) to robustly answer multistep questions by generating and answering sub-questions. We propose Chain-of-Questions, a framework that trains a model to generate sub-questions and sub-answers one at a time by leveraging human annotated question decomposition meaning representation (QDMR). The key technical challenge is that QDMR only contains sub-questions but not answers to those sub-questions, so we treat sub-answers as latent variables and optimize them using a novel dynamic mixture of Hard-EM and MAPO. Chain-of-Questions greatly outperforms strong neuro-symbolic methods by 9.0 F1 on DROP contrast set, and outperforms GPT-3.5 by 24.3 F1 on HOTPOTQA adversarial set, thus demonstrating the effectiveness and robustness of our framework.