Lotus: Creating Short Videos From Long Videos With Abstractive and Extractive Summarization
This addresses the problem for video creators on platforms like TikTok and Instagram who struggle with manually planning and editing clips from long videos, though it is incremental as it builds on existing summarization approaches.
The authors tackled the challenge of repurposing long-form videos into short-form content by developing Lotus, a system that combines abstractive and extractive summarization to generate short videos, and in a user study, it was compared to existing practices and an extractive baseline method.
Short-form videos are popular on platforms like TikTok and Instagram as they quickly capture viewers' attention. Many creators repurpose their long-form videos to produce short-form videos, but creators report that planning, extracting, and arranging clips from long-form videos is challenging. Currently, creators make extractive short-form videos composed of existing long-form video clips or abstractive short-form videos by adding newly recorded narration to visuals. While extractive videos maintain the original connection between audio and visuals, abstractive videos offer flexibility in selecting content to be included in a shorter time. We present Lotus, a system that combines both approaches to balance preserving the original content with flexibility over the content. Lotus first creates an abstractive short-form video by generating both a short-form script and its corresponding speech, then matching long-form video clips to the generated narration. Creators can then add extractive clips with an automated method or Lotus's editing interface. Lotus's interface can be used to further refine the short-form video. We compare short-form videos generated by Lotus with those using an extractive baseline method. In our user study, we compare creating short-form videos using Lotus to participants' existing practice.