Semantic-Guided Unsupervised Video Summarization
This work addresses video summarization for efficient browsing of multimedia content on social platforms, representing an incremental improvement over existing GAN-based methods.
The paper tackled the problem of unstable training and lack of semantic guidance in unsupervised video summarization by proposing a semantic-guided method with a frame-level semantic alignment attention mechanism and incremental training strategy, achieving superior performance on multiple benchmark datasets.
Video summarization is a crucial technique for social understanding, enabling efficient browsing of massive multimedia content and extraction of key information from social platforms. Most existing unsupervised summarization methods rely on Generative Adversarial Networks (GANs) to enhance keyframe selection and generate coherent, video summaries through adversarial training. However, such approaches primarily exploit unimodal features, overlooking the guiding role of semantic information in keyframe selection, and often suffer from unstable training. To address these limitations, we propose a novel Semantic-Guided Unsupervised Video Summarization method. Specifically, we design a novel frame-level semantic alignment attention mechanism and integrate it into a keyframe selector, which guides the Transformer-based generator within the adversarial framework to better reconstruct videos. In addition, we adopt an incremental training strategy to progressively update the model components, effectively mitigating the instability of GAN training. Experimental results demonstrate that our approach achieves superior performance on multiple benchmark datasets.