Improving Factual Consistency of Abstractive Summarization on Customer Feedback
This work addresses factual consistency issues in summarization for e-commerce sellers, but it is incremental as it builds on existing models with targeted improvements.
The paper tackled the problem of factual errors in abstractive summarization of customer feedback, specifically wrong entity detection and incorrect product-defect descriptions, by introducing data augmentation with corrupted summaries and a contrastive loss, resulting in a large reduction of these errors for models like BART and T5.
E-commerce stores collect customer feedback to let sellers learn about customer concerns and enhance customer order experience. Because customer feedback often contains redundant information, a concise summary of the feedback can be generated to help sellers better understand the issues causing customer dissatisfaction. Previous state-of-the-art abstractive text summarization models make two major types of factual errors when producing summaries from customer feedback, which are wrong entity detection (WED) and incorrect product-defect description (IPD). In this work, we introduce a set of methods to enhance the factual consistency of abstractive summarization on customer feedback. We augment the training data with artificially corrupted summaries, and use them as counterparts of the target summaries. We add a contrastive loss term into the training objective so that the model learns to avoid certain factual errors. Evaluation results show that a large portion of WED and IPD errors are alleviated for BART and T5. Furthermore, our approaches do not depend on the structure of the summarization model and thus are generalizable to any abstractive summarization systems.