"I'd rather just go to bed": Understanding Indirect Answers
This addresses a key challenge in making dialog systems more natural by understanding indirect responses, but it is incremental as it builds on existing language models and datasets.
The authors tackled the problem of interpreting indirect answers in dialog by creating the first large-scale English corpus 'Circa' with 34,268 question-answer pairs, and developed BERT-based models that achieved 82-88% accuracy for 4-class and 74-85% for 6-class distinctions, though performance was insufficient for robust dialog.
We revisit a pragmatic inference problem in dialog: understanding indirect responses to questions. Humans can interpret 'I'm starving.' in response to 'Hungry?', even without direct cue words such as 'yes' and 'no'. In dialog systems, allowing natural responses rather than closed vocabularies would be similarly beneficial. However, today's systems are only as sensitive to these pragmatic moves as their language model allows. We create and release the first large-scale English language corpus 'Circa' with 34,268 (polar question, indirect answer) pairs to enable progress on this task. The data was collected via elaborate crowdsourcing, and contains utterances with yes/no meaning, as well as uncertain, middle-ground, and conditional responses. We also present BERT-based neural models to predict such categories for a question-answer pair. We find that while transfer learning from entailment works reasonably, performance is not yet sufficient for robust dialog. Our models reach 82-88% accuracy for a 4-class distinction, and 74-85% for 6 classes.