Ask an Expert: Leveraging Language Models to Improve Strategic Reasoning in Goal-Oriented Dialogue Models
This addresses the issue of unreliable dialogue responses in goal-oriented tasks, particularly for mental health applications, though it is incremental as it builds on existing LLM and dialogue model techniques.
The paper tackles the problem of dialogue models generating poor responses in underrepresented scenarios by proposing an 'Ask an Expert' framework where models consult an LLM expert for advice, resulting in a ~10% improvement over baselines and near-human scores on engagement and helpfulness metrics in a mental health support domain.
Existing dialogue models may encounter scenarios which are not well-represented in the training data, and as a result generate responses that are unnatural, inappropriate, or unhelpful. We propose the "Ask an Expert" framework in which the model is trained with access to an "expert" which it can consult at each turn. Advice is solicited via a structured dialogue with the expert, and the model is optimized to selectively utilize (or ignore) it given the context and dialogue history. In this work the expert takes the form of an LLM. We evaluate this framework in a mental health support domain, where the structure of the expert conversation is outlined by pre-specified prompts which reflect a reasoning strategy taught to practitioners in the field. Blenderbot models utilizing "Ask an Expert" show quality improvements across all expert sizes, including those with fewer parameters than the dialogue model itself. Our best model provides a $\sim 10\%$ improvement over baselines, approaching human-level scores on "engingingness" and "helpfulness" metrics.