State Space Models for Extractive Summarization in Low Resource Scenarios
This work addresses extractive summarization for low-resource scenarios, but it appears incremental as it builds on existing state space models and compression techniques.
The paper tackles extractive summarization in low-resource settings by proposing MPoincareSum, which uses a Mamba state space model and Poincare compression to select relevant sentences, and it outperforms existing methods on the Amazon review dataset as measured by ROUGE scores.
Extractive summarization involves selecting the most relevant sentences from a text. Recently, researchers have focused on advancing methods to improve state-of-the-art results in low-resource settings. Motivated by these advancements, we propose the MPoincareSum method. This method applies the Mamba state space model to generate the semantics of reviews and sentences, which are then concatenated. A Poincare compression is used to select the most meaningful features, followed by the application of a linear layer to predict sentence relevance based on the corresponding review. Finally, we paraphrase the relevant sentences to create the final summary. To evaluate the effectiveness of MPoincareSum, we conducted extensive experiments using the Amazon review dataset. The performance of the method was assessed using ROUGE scores. The experimental results demonstrate that MPoincareSum outperforms several existing approaches in the literature