Automated Conversion of Music Videos into Lyric Videos
This work addresses the time-consuming task of creating lyric videos for musicians and fans, but it is incremental as it builds on prior work and guidelines.
The authors tackled the problem of automatically converting music videos into lyric videos by proposing design guidelines for readability and unified attention, and implemented a fully automated pipeline that demonstrated robustness across diverse inputs and effectiveness in a user study.
Musicians and fans often produce lyric videos, a form of music videos that showcase the song's lyrics, for their favorite songs. However, making such videos can be challenging and time-consuming as the lyrics need to be added in synchrony and visual harmony with the video. Informed by prior work and close examination of existing lyric videos, we propose a set of design guidelines to help creators make such videos. Our guidelines ensure the readability of the lyric text while maintaining a unified focus of attention. We instantiate these guidelines in a fully automated pipeline that converts an input music video into a lyric video. We demonstrate the robustness of our pipeline by generating lyric videos from a diverse range of input sources. A user study shows that lyric videos generated by our pipeline are effective in maintaining text readability and unifying the focus of attention.