Pseudo Labeling and Negative Feedback Learning for Large-scale Multi-label Domain Classification
This work addresses the challenge of handling multiple overlapping domains in conversational systems, but it is incremental as it builds on existing pseudo labeling and feedback techniques.
The paper tackles the problem of limited ground-truth labels in large-scale multi-label domain classification by using pseudo labeling and negative feedback learning to improve model performance, showing significant improvements in domain classification with hypothesis reranking on user utterances from an intelligent conversational system.
In large-scale domain classification, an utterance can be handled by multiple domains with overlapped capabilities. However, only a limited number of ground-truth domains are provided for each training utterance in practice while knowing as many as correct target labels is helpful for improving the model performance. In this paper, given one ground-truth domain for each training utterance, we regard domains consistently predicted with the highest confidences as additional pseudo labels for the training. In order to reduce prediction errors due to incorrect pseudo labels, we leverage utterances with negative system responses to decrease the confidences of the incorrectly predicted domains. Evaluating on user utterances from an intelligent conversational system, we show that the proposed approach significantly improves the performance of domain classification with hypothesis reranking.