Domain Agnostic Real-Valued Specificity Prediction
This work addresses the need for more detailed and domain-agnostic specificity prediction to enhance downstream applications like dialogue generation, though it is incremental as it builds on existing unsupervised adaptation methods.
The paper tackled the problem of predicting nuanced real-valued sentence specificity across domains without labeled data, achieving a 50% to 68% reduction in mean absolute error compared to state-of-the-art domain-specific systems.
Sentence specificity quantifies the level of detail in a sentence, characterizing the organization of information in discourse. While this information is useful for many downstream applications, specificity prediction systems predict very coarse labels (binary or ternary) and are trained on and tailored toward specific domains (e.g., news). The goal of this work is to generalize specificity prediction to domains where no labeled data is available and output more nuanced real-valued specificity ratings. We present an unsupervised domain adaptation system for sentence specificity prediction, specifically designed to output real-valued estimates from binary training labels. To calibrate the values of these predictions appropriately, we regularize the posterior distribution of the labels towards a reference distribution. We show that our framework generalizes well to three different domains with 50%~68% mean absolute error reduction than the current state-of-the-art system trained for news sentence specificity. We also demonstrate the potential of our work in improving the quality and informativeness of dialogue generation systems.