Bias in Opinion Summarisation from Pre-training to Adaptation: A Case Study in Political Bias
This addresses bias in summarization for users of social media and opinion analysis, but it is incremental as it builds on prior work on bias in extractive models.
The study tackled bias in abstractive opinion summarization by quantifying political bias across models and adaptation methods, finding that most models exhibit intrinsic bias and that tuning fewer parameters reduces bias but depends on training data diversity.
Opinion summarisation aims to summarise the salient information and opinions presented in documents such as product reviews, discussion forums, and social media texts into short summaries that enable users to effectively understand the opinions therein. Generating biased summaries has the risk of potentially swaying public opinion. Previous studies focused on studying bias in opinion summarisation using extractive models, but limited research has paid attention to abstractive summarisation models. In this study, using political bias as a case study, we first establish a methodology to quantify bias in abstractive models, then trace it from the pre-trained models to the task of summarising social media opinions using different models and adaptation methods. We find that most models exhibit intrinsic bias. Using a social media text summarisation dataset and contrasting various adaptation methods, we find that tuning a smaller number of parameters is less biased compared to standard fine-tuning; however, the diversity of topics in training data used for fine-tuning is critical.