Learning End-to-End Goal-Oriented Dialog
This work addresses the problem of reducing handcrafting in goal-oriented dialog systems for developers, though it is incremental as it builds on existing end-to-end methods.
The paper tackles the challenge of scaling goal-oriented dialog systems by proposing a testbed for evaluating end-to-end approaches, showing that a Memory Networks-based system achieves promising but imperfect performance, with results comparable to a hand-crafted baseline on benchmark data.
Traditional dialog systems used in goal-oriented applications require a lot of domain-specific handcrafting, which hinders scaling up to new domains. End-to-end dialog systems, in which all components are trained from the dialogs themselves, escape this limitation. But the encouraging success recently obtained in chit-chat dialog may not carry over to goal-oriented settings. This paper proposes a testbed to break down the strengths and shortcomings of end-to-end dialog systems in goal-oriented applications. Set in the context of restaurant reservation, our tasks require manipulating sentences and symbols, so as to properly conduct conversations, issue API calls and use the outputs of such calls. We show that an end-to-end dialog system based on Memory Networks can reach promising, yet imperfect, performance and learn to perform non-trivial operations. We confirm those results by comparing our system to a hand-crafted slot-filling baseline on data from the second Dialog State Tracking Challenge (Henderson et al., 2014a). We show similar result patterns on data extracted from an online concierge service.