Leveraging Audio Gestalt to Predict Media Memorability
This work is significant for content curators and creators seeking to produce more memorable media, by providing a method to predict video memorability.
This paper addresses the task of automatically predicting video memorability. The authors developed a multimodal deep learning approach using late fusion of visual, semantic, and auditory features, with audio gestalt guiding the optimal feature combination.
Memorability determines what evanesces into emptiness, and what worms its way into the deepest furrows of our minds. It is the key to curating more meaningful media content as we wade through daily digital torrents. The Predicting Media Memorability task in MediaEval 2020 aims to address the question of media memorability by setting the task of automatically predicting video memorability. Our approach is a multimodal deep learning-based late fusion that combines visual, semantic, and auditory features. We used audio gestalt to estimate the influence of the audio modality on overall video memorability, and accordingly inform which combination of features would best predict a given video's memorability scores.