Domain Aligned Prefix Averaging for Domain Generalization in Abstractive Summarization
This addresses the problem of domain generalization for abstractive summarization, offering a computationally efficient solution for researchers and practitioners, though it is incremental in nature.
The paper tackles domain generalization in abstractive summarization by proposing a lightweight weight averaging method called Domain Aligned Prefix Averaging, which achieves comparable or better performance than baselines across four diverse domains.
Domain generalization is hitherto an underexplored area applied in abstractive summarization. Moreover, most existing works on domain generalization have sophisticated training algorithms. In this paper, we propose a lightweight, weight averaging based, Domain Aligned Prefix Averaging approach to domain generalization for abstractive summarization. Given a number of source domains, our method first trains a prefix for each one of them. These source prefixes generate summaries for a small number of target domain documents. The similarity of the generated summaries to their corresponding documents is used for calculating weights required to average source prefixes. In DAPA, prefix tuning allows for lightweight finetuning, and weight averaging allows for the computationally efficient addition of new source domains. When evaluated on four diverse summarization domains, DAPA shows comparable or better performance against the baselines, demonstrating the effectiveness of its prefix averaging scheme.