A Cascaded Architecture for Extractive Summarization of Multimedia Content via Audio-to-Text Alignment
This work addresses multimedia summarization for improved information retrieval and accessibility, but it is incremental as it integrates existing tools and models.
The study tackled the problem of extracting key insights from multimedia content like YouTube videos by proposing a cascaded architecture for extractive summarization via audio-to-text alignment, and it demonstrated improved performance over conventional methods using ROUGE and F1 scores.
This study presents a cascaded architecture for extractive summarization of multimedia content via audio-to-text alignment. The proposed framework addresses the challenge of extracting key insights from multimedia sources like YouTube videos. It integrates audio-to-text conversion using Microsoft Azure Speech with advanced extractive summarization models, including Whisper, Pegasus, and Facebook BART XSum. The system employs tools such as Pytube, Pydub, and SpeechRecognition for content retrieval, audio extraction, and transcription. Linguistic analysis is enhanced through named entity recognition and semantic role labeling. Evaluation using ROUGE and F1 scores demonstrates that the cascaded architecture outperforms conventional summarization methods, despite challenges like transcription errors. Future improvements may include model fine-tuning and real-time processing. This study contributes to multimedia summarization by improving information retrieval, accessibility, and user experience.