Audio Match Cutting: Finding and Creating Matching Audio Transitions in Movies and Videos
This addresses a specific video editing challenge for filmmakers and editors, offering an incremental improvement by automating a previously manual task.
The paper tackles the problem of automatically finding and creating 'audio match cuts' in videos, where audio from different shots blends smoothly, by developing a self-supervised representation and a coarse-to-fine pipeline that recommends matching shots and blends audio, achieving results demonstrated in examples on a project page.
A "match cut" is a common video editing technique where a pair of shots that have a similar composition transition fluidly from one to another. Although match cuts are often visual, certain match cuts involve the fluid transition of audio, where sounds from different sources merge into one indistinguishable transition between two shots. In this paper, we explore the ability to automatically find and create "audio match cuts" within videos and movies. We create a self-supervised audio representation for audio match cutting and develop a coarse-to-fine audio match pipeline that recommends matching shots and creates the blended audio. We further annotate a dataset for the proposed audio match cut task and compare the ability of multiple audio representations to find audio match cut candidates. Finally, we evaluate multiple methods to blend two matching audio candidates with the goal of creating a smooth transition. Project page and examples are available at: https://denfed.github.io/audiomatchcut/