Towards a Universal NLG for Dialogue Systems and Simulators with Future Bridging
This addresses the need for more efficient and reusable NLG in dialogue systems, though it appears incremental as it builds on existing data-driven trends.
The paper tackles the problem of limited training data and lack of reusability in dialogue NLG by proposing a future bridging NLG (FBNLG) concept that uses future utterances for self-supervised training, enabling application in various dialogue scenarios with minimal adaptation.
In a dialogue system pipeline, a natural language generation (NLG) unit converts the dialogue direction and content to a corresponding natural language realization. A recent trend for dialogue systems is to first pre-train on large datasets and then fine-tune in a supervised manner using datasets annotated with application-specific features. Though novel behaviours can be learned from custom annotation, the required effort severely bounds the quantity of the training set, and the application-specific nature limits the reuse. In light of the recent success of data-driven approaches, we propose the novel future bridging NLG (FBNLG) concept for dialogue systems and simulators. The critical step is for an FBNLG to accept a future user or system utterance to bridge the present context towards. Future bridging enables self supervised training over annotation-free datasets, decoupled the training of NLG from the rest of the system. An FBNLG, pre-trained with massive datasets, is expected to apply in classical or new dialogue scenarios with minimal adaptation effort. We evaluate a prototype FBNLG to show that future bridging can be a viable approach to a universal few-shot NLG for task-oriented and chit-chat dialogues.