Soundify: Matching Sound Effects to Video
This addresses the problem of sound editing for video editors, offering a tool to enhance workflow efficiency, though it appears incremental as it builds on existing audio-visual matching techniques.
The paper tackles the challenge of matching sound effects to video by introducing Soundify, a system that identifies, synchronizes, and adjusts sounds for spatial audio, resulting in reduced task completion time and improved usability in expert studies.
In the art of video editing, sound helps add character to an object and immerse the viewer within a space. Through formative interviews with professional editors (N=10), we found that the task of adding sounds to video can be challenging. This paper presents Soundify, a system that assists editors in matching sounds to video. Given a video, Soundify identifies matching sounds, synchronizes the sounds to the video, and dynamically adjusts panning and volume to create spatial audio. In a human evaluation study (N=889), we show that Soundify is capable of matching sounds to video out-of-the-box for a diverse range of audio categories. In a within-subjects expert study (N=12), we demonstrate the usefulness of Soundify in helping video editors match sounds to video with lighter workload, reduced task completion time, and improved usability.