Reducing Sensitivity on Speaker Names for Text Generation from Dialogues
This addresses fairness issues in dialogue-processing applications where speaker name changes shouldn't affect outputs, though it appears incremental as it builds on known methods.
The paper tackles the problem of pre-trained language models being sensitive to speaker name changes in dialogue text generation, which can cause unfairness. It proposes a quantitative measurement method and benchmarks multiple approaches, showing their novel method achieves favorable performance in reducing sensitivity while maintaining generation quality.
Changing speaker names consistently throughout a dialogue should not affect its meaning and corresponding outputs for text generation from dialogues. However, pre-trained language models, serving as the backbone for dialogue-processing tasks, have shown to be sensitive to nuances. This may result in unfairness in real-world applications. No comprehensive analysis of this problem has been done in the past. In this work, we propose to quantitatively measure a model's sensitivity on speaker names, and comprehensively evaluate a number of known methods for reducing speaker name sensitivity, including a novel approach of our own. Extensive experiments on multiple datasets provide a benchmark for this problem and show the favorable performance of our approach in sensitivity reduction and quality of generation.