Towards Handling Unconstrained User Preferences in Dialogue
This work addresses the need for more natural dialogue interfaces in information navigation, such as venue search, by allowing users to specify preferences beyond rigid database constraints, though it is incremental as it builds on existing retrieval and classification methods.
The paper tackled the problem of handling unconstrained user preferences in dialogue systems, which are typically limited by predefined database schemas, by using information retrieval from unstructured knowledge to identify relevant entities, achieving a weighted F1 score of 0.856 with a supervised classifier.
A user input to a schema-driven dialogue information navigation system, such as venue search, is typically constrained by the underlying database which restricts the user to specify a predefined set of preferences, or slots, corresponding to the database fields. We envision a more natural information navigation dialogue interface where a user has flexibility to specify unconstrained preferences that may not match a predefined schema. We propose to use information retrieval from unstructured knowledge to identify entities relevant to a user request. We update the Cambridge restaurants database with unstructured knowledge snippets (reviews and information from the web) for each of the restaurants and annotate a set of query-snippet pairs with a relevance label. We use the annotated dataset to train and evaluate snippet relevance classifiers, as a proxy to evaluating recommendation accuracy. We show that with a pretrained transformer model as an encoder, an unsupervised/supervised classifier achieves a weighted F1 of .661/.856.