ReFeed: Multi-dimensional Summarization Refinement with Reflective Reasoning on Feedback
This work addresses the problem of improving summarization quality across multiple dimensions for NLP researchers and practitioners, presenting an incremental advancement through a new dataset and refinement method.
The paper tackles the challenge of multi-dimensional summarization refinement by introducing ReFeed, a pipeline that uses reflective reasoning on feedback to enhance multiple dimensions, and releases a dataset called SumFeed-CoT for training lightweight models, with experiments showing improved performance in handling trade-offs between dimensions and robustness to noisy feedback.
Summarization refinement faces challenges when extending to multi-dimension. In this paper, we introduce ReFeed, a powerful summarization refinement pipeline that enhances multiple dimensions through reflective reasoning on feedback. To achieve this, we release SumFeed-CoT, a large-scale Long-CoT-based dataset optimized for training a lightweight model with reflective reasoning. Our experiments reveal how the number of dimensions, feedback exposure, and reasoning policy influence refinement performance, highlighting reflective reasoning and simultaneously addressing multiple feedback is crucial to mitigate trade-off between dimensions. Furthermore, ReFeed is robust to noisy feedback and feedback order. Lastly, our finding emphasizes that creating data with a proper goal and guideline constitutes a fundamental pillar of effective reasoning. The dataset and model will be released.